1. 80
    I want extern "fil-c" domenkozar.com
  1.  

    1. 29

      I love this idea.

      I think it’s possible to create a Rust-Fil-C integration where everything that’s currently unsafe in Rust is governed by Fil-ABI.

      It might require dialect-level changes on both the Rust and Fil-C side.

      I’ve posted about it on X. I should sit down and write more about it at some point. I would be especially happy to brain dump what I understand about this to anyone who is interested in dedicating time to prototyping this.

      1. 15

        I've seen contradictings claims about the possibility of a hybrid fil/native program, so I'll believe it when I see it. FWIW, Zig seems to be going for whole-program fil.

        If Rust manages a hybrid mode, it'll be at an FFI boundary (like the article suggests), not for "everything that's currently unsafe". Because unsafe Rust has a smaller granularity than what fil can work on, and the errors that fil can detect are likely to happen outside the unsafe block that caused them. At that stage, you're basically looking at whole-program fil, which doesn't make much sense for Rust.

        1. 3

          I think you're right about unsafe granularity, but I want to add that I think whole-program fil for Rust does make sense as an idea. It could serve as something like compiled/fast Miri.

          1. 11

            Just now I happened to see that rustc_codegen_jvm recently added raw pointer support, and apparently it does a pretty good job at catching UB. Might be a good alternative, not sure how the perf is compared to a hypothetical Fil-Rust.

            1. 6

              Have you heard about BorrowSanitizer? That's what I'm personally betting on in the compiled/fast Miri category, though I'm not sure how usable it is at the moment.

              1. 3

                Can you think of a Rust bug that Miri doesn't catch but Fil would ? I'm not saying it's impossible, but I haven't seen any example yet, and my guess is that it would be academic but not realistic. If miri finds more bugs, that's what people will test with, even if it's slower.

                The only reason for whole-prog Fil would then be that you want Fil for the C parts, but FFI-Fil doesn't work or is too cumbersome for some reason ?

                1. 5

                  Can you think of a Rust bug that Miri doesn't catch but Fil would ?

                  Specifically Rust bugs, probably not, since Miri is intended to catch all Rust language UB. That being said, the lack of FFI support is a notable current limitation in Miri with practical implications that a hypothetical Fil-Rust could help with, though to be fair there is work being done to Miri to better support FFI.

                  If miri finds more bugs, that's what people will test with, even if it's slower.

                  I think it's going to depend on what tests you have in mind. Fuzzing, for instance, is going to be quite a bit more sensitive to slowdowns.

                  1. 3

                    That being said, the lack of FFI support is a notable current limitation in Miri with practical implications that a hypothetical Fil-Rust could help with

                    Sure, but we're back to "Fil is interesting at the FFI boundary", not for the whole rust-and-c program.

                    Fuzzing, for instance, is going to be quite a bit more sensitive to slowdowns.

                    Fair point.

                    1. 2

                      Sure, but we're back to "Fil is interesting at the FFI boundary", not for the whole rust-and-c program.

                      Hrm, fair. Maybe there could be some interesting cross-language optimizations that could be done? Fil-C is mostly an LLVM pass so I could imagine it working after both Rust and C sources have been lowered to LLVM IR.

                    2. 1

                      I'd go a step further and say "FFI support is a notable current limitation of Rust" 😕

                      After having hand-written a few dozen thousand lines of it it's still the biggest downside. The original thesis of Rust is not very friendly to anything that lives outside of the compilers purview.

                      Very much looking forward to any and all improvements here.

                      1. 2

                        Where does (c)bindgen fail you, that you feel the need to hand-write FFI ? Or in what way do you think Rust could be friendlier to languages it doesn't control ?

                        1. 1

                          One of the biggest problems I personally face is the lack of dll import/export style constructs. Such that it's hard to share the same global statics across modules, even within rusts own walls.

                          Lack of ABI stability is also a huge issue.

                          The other big one is just what we're already talking about, you can never /really/ trust what happens when you call out to a different language thus it always being unsafe. But not only does the code on the other side have to work well enough, the actual handoff itself can be somewhat tricky. Like codifying what type of guarantees the API has must really be rock solid.

              2. 3

                I'd happy to invite you to https://oceansprint.org/ in 2027 that I organize for geeks to socialize for a week and someone from Rust community that wants to pair up.

                1. 2

                  This sounds great and oceansprint is soon; but I am also selfish and would want us to hack on it for https://tacosprint.org/ next summer lol

                  1. 2

                    Happy to also do it in Mexico :)

                    1. 1

                      Application deadline: May 15, 2026

                      Oh :(

                      1. 3

                        We are doing it again 2027! It was amazing. Check my blog for a sprint report :)

                2. 20

                  I don't know how to move beyond this proposed minimal ABI: Fil-C needs to recompile the entire program in order to track capabilities (which is why it doesn't work on e.g. Windows), so you can't really pass pointers back and forth without having Rust run under Fil-C.

                  Memory-safety violations panic instead of becoming exploits.

                  Well, some of them. Fil-C turns undefined behavior into panics or legal behavior (e.g. uninitialized objects are zero-initialized instead). But Fil-C is a full C implementation, and so allows all the memory safety violations that are legal in C - including out of bounds & use-after frees, as long as they happen entirely within one object.

                  There are also the cases like unsynchronized accesses to the same object from different threads, where Fil-C just says "Yeah this isn't UB anymore" but doesn't actually fix the memory safety issue.

                  1. 4

                    In Fil-C calling free() is optional as it uses a GC to actually reclaim unreachable memory, but if you do call free() then it updates the capability so that use-after-free and double-free are guaranteed to be caught.

                    1. 13

                      Yes, but if you e.g. have a bump allocator and free its contents by resetting the bump pointer to the start so you can use it for a new set of allocations, Fil-C will still let you read from and write to the old allocations long after they've been overwritten because in C that's perfectly legal.

                      1. 1

                        If you make an arena then of course you are responsible for managing it. That's your take on memory allocation, not anything to do with what is provided by or visible to Fil-C. You're not freeing the arena, so it's not something Fil-C can or even should know about.

                        1. 7

                          Yes, and that's my point.

                    2. 1

                      That's why it's so appealing because of https://github.com/mbrock/filnix

                      1. 2

                        Not sure how this related to my comment?

                        1. 1

                          Sorry, I wasn't clear enough.

                          Without pointers, there's no modifications need to Rust.

                          While for pointers you'd need fil-c integrated into rust and using the Nix machinery we can then distribute that.

                      2. 1

                        But Fil-C is a full C implementation, and so allows all the memory safety violations that are legal in C - including out of bounds & use-after frees, as long as they happen entirely within one object.

                        I'm almost certain that what you wrote here was not what you intended to say. Out of bounds accesses are not legal in C whether they are constrained to a single object or not. Use-after-frees are never legal. Both yield undefined behaviour.

                        I guess from your followup comment below that by "use-after-free" you actually mean reusing storage for a new object (without calling free at all), which is legal, yes, but it's not what is normally meant by use-after-free. Out-of-bounds access is always illegal.

                        1. 9

                          What I mean is that if, for example, a function takes a value by reference, which can look like a normal object being passed in Java/Python/JavaScript/… (languages where references point at objects) or with an explicit reference in Go/Rust/Vale/… (languages where references point at a location). In these languages, the function can only access in bounds of that location.

                          But in C, you can legally access past the bounds of the specific pointed-to subobject, and this is very common to do (of course if you end up violating type safety along the way it can be UB, but Fil-C doesn't catch that either).

                          A simple example of this posted by tstack in earlier discussions:

                          #include <stdlib.h>
                          #include <stdio.h>
                          #include <string.h>
                          
                          struct User {
                              char name[8];
                              int is_root;
                          };
                          
                          int main(int argc, char*argv[]) {
                              struct User* user = malloc(sizeof(struct User));
                              strcpy(user->name, argv[1]);
                              if (user->is_root) {
                                  printf("User is root!\n");
                              }
                              return 0;
                          }
                          

                          Compile with filcc, then run it:

                          > ./fillo AzureDiamond 
                          User is root!
                          
                          1. 5

                            As I've mentioned previously, that is not what Fil-C is trying to address, different hardening technologies address it - specifically FORTIFY_SOURCE. i.e. if that program is compiled under gcc, with FORTIFY_SOURCE, the access is trapped.

                            The only issue at the moment for Fil-C is that it is based on clang, and the clang implementation of FORTIFY_SOURCE can not yet handle that case; but that can be addressed.

                            See these two posts:

                            https://lobste.rs/s/x7jtkt/memory_safety_absolutists#c_v4tfxu

                            https://lobste.rs/s/x7jtkt/memory_safety_absolutists#c_0xb5sy

                            1. 2

                              But in C, you can legally access past the bounds of the specific pointed-to subobject

                              No, you cannot legally do that in the manner you seem to be describing. It is undefined behaviour to do so. In C99 (the reference I happen to have on hand), chapter 7.21.1 (String function conventions): in all cases a char * or void * argument points to the initial (lowest addressed) character of the array. If an array is accessed beyond the end of an object, the behavior is undefined.

                              Compile with filcc, then run it

                              ... and you will see one possible behaviour; this proves nothing about whether it is legal in C, in general, because UB is involved. (It happens that Fil-C tightens the semantics).

                              In any case "out of bounds access" refers to many other situations other than using string or memory copying functions. It is certainly not legal to attempt access to an array element by an index that is out of bounds of the array, even if the element would (if it existed) correspond to another field in the same containing object. In general, out of bounds access is not legal, and if the example you gave were legal, it would be a special case.

                              1. 11

                                No, you cannot legally do that in the manner you seem to be describing.

                                There’s an extra step in the logic.

                                If you are given a pointer to a member of a struct, it’s legal to use something like Linux’s container_of() macro to access other members in the same struct. Therefore Fil-C does not try to catch pointer shenanigans within an allocation; it only traps when the program strays outside an allocation, not necessarily when it accesses an array out of bounds.

                                The program above is an example of UB that Fil-C fails to catch, not an example of the kind of valid accesses that are technically out-of-bounds. The latter would be something like,

                                int user_is_root(char *name) {
                                    User *user = container_of(name, User, name);
                                    // access is out of bounds of name
                                    return user->is_root;
                                }
                                
                                1. 2

                                  There’s an extra step in the logic. ... If you are given a pointer to a member of a struct, it’s legal to use something like Linux’s container_of() macro to access other members in the same struct

                                  Doing such would not be considered an "out of bounds" access, and was not what the example that I commented on was doing. As I said:

                                  No, you cannot legally do that in the manner you seem to be describing.

                                  // access is out of bounds of name
                                  

                                  While the access is outside the bounds of name, it wouldn't constitute what would be described as an out-of-bounds access; that term normally refers to the logic error.

                                  1. 8

                                    Right, I was clarifying that what mira intended to describe is not what you thought they described: they were talking about out-of-bounds accesses in a general multi-language sense, not the C rules-lawyering sense you are using; you are still misinterpreting their program as supposedly an example of the kind of out-of-bounds access that C allows, but as I explained, that is not the point the program was illustrating.

                                    1. 5

                                      It's a mix of both – I was indeed misremembering some parts of the C standard (in particular the rule found as 6.5.7.9 in the latest C23 working draft), and Fil-C's marketing helped me make my mistake. Thank you both for clearing this up ^^

                                      I'm curious about why container_of is valid - or maybe it isn't according to the standard, but gcc defines behavior for it?

                                      1. 3

                                        You are most welcome. I haven't looked at the specific container_of implementation that Linux uses, but the best-sanctioned variation that I know is to cast the pointer-to-member into an integer and subtract the offset of the member before casting back to the pointer-to-container type. Because you're not doing pointer arithmetic directly you avoid the UB that you normally get from pointer arithmetic which moves outside the bounds of the pointed-to (sub)object.

                                        Technically the result is implementation-defined (casting from integer to pointer and vice versa is implementation-defined) but in practice on typical architectures/compilers it works as it's intended to.

                                        Often casting to (char *) instead of an integer also works because compilers tend to relax the rules for char *. But, it's more likely to be UB (some might say it is UB, I think the language standard isn't completely clear on this).

                                      2. 3

                                        The original quote was:

                                        But Fil-C is a full C implementation, and so allows all the memory safety violations that are legal in C - including out of bounds & use-after frees, as long as they happen entirely within one object

                                        What you are talking about is not a memory safety violation even if it is an out-of-bounds access "in some sense". If mira meant what you are saying she meant, then she nevertheless had it wrong when she said "memory safety violations are legal in C"; additionally, the code example she gave did not illustrate what she thought it did.

                                        you are still misinterpreting their program as supposedly an example of the kind of out-of-bounds access that C allows, but as I explained, that is not the point the program was illustrating.

                                        From that post:

                                        But in C, you can legally access past the bounds of the specific pointed-to subobject [...] A simple example of this posted by tstack in earlier discussions:

                                        This was the context that example was presented in, that I responded to. It does quite clearly claim that the example is of the kind of out-of-bounds access that C allows.

                                        1. 7

                                          What you are talking about is not a memory safety violation even if it is an out-of-bounds access "in some sense".

                                          But it is a memory safety violation. Consider a safe implementation that keeps tight bounds on the name array that is passed to user_is_root(): the function would crash at runtime with an out-of-bounds access. Obviously (which is the point of the discussion) this implementation would be too strict in practice, hence Fil-C does bounds checking on allocations not bounds checking on arrays. And the point of the example in mira’s post is that this looseness matters, because a C runtime can’t tell the difference between “good” out of bounds and “bad” out of bounds at this granularity.

                                          And it’s subtle whether that strcpy() is UB or not, since strcpy() accesses its destination as bytes, which is legal for any object. So these should all be equivalent:

                                          strcpy(user->name, argv[1]);
                                          // inline strcpy
                                          memcpy(user->name, argv[1], strlen(argv[1]) + 1);
                                          // (void*) struct is same as (void*) first member
                                          memcpy(user, argv[1], strlen(argv[1]) + 1);
                                          
                                          1. 1

                                            But it is a memory safety violation.

                                            I disagree.

                                            Consider a safe implementation that keeps tight bounds on the name array that is passed to user_is_root(): the function would crash at runtime with an out-of-bounds access

                                            Obviously (which is the point of the discussion) this implementation would be too strict in practice, hence Fil-C does bounds checking on allocations not bounds checking on arrays

                                            Too strict in practice, yes - because it prevents valid accesses. (But, I suspect checking allocation bounds rather than individual subobject bounds is also better for performance).

                                            Consider an implementation that doesn't keep such bounds, and which defines reasonable conversion from integer to pointer as required for a reasonable implementation of container_of. No undefined behaviour is needed for container_of to function correctly, and it's not accessing memory outside of what it is supposed to; there's no safety violation.

                                            And it’s subtle whether that strcpy() is UB or not, since strcpy() accesses its destination as bytes, which is legal for any object

                                            The arguments are required to be char arrays, and the text states that "If an array is accessed beyond the end of an object, the behavior is undefined". The example clearly does so. I think you're confusing the strict aliasing rule (which does have special allowances for access via char) with the requirements on the string functions.

                                            1. 3

                                              The arguments are required to be char arrays, and the text states that "If an array is accessed beyond the end of an object, the behavior is undefined".

                                              It doesn’t say that they have to be declared as char arrays, they can be “other objects treated as arrays of character type”. There is also wording earlier in the standard that allows any object to be accessed as a character array, but is a bit roundabout: 6.3.3.3 Conversions - Pointers says:

                                              When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers to the remaining bytes of the object.

                                              But incrementing a pointer is only defined for arrays (6.5.7 Additive operators) so putting it together we have to conclude that an expression designating a character array might refer to any type of object or part thereof. Which also implies that the bounds of the array are indeterminate, because it isn’t clear which subobject the pointer is supposed to refer to.

                                              (Perhaps this looseness only applies to structures and their first members (6.7.3.2 “A pointer to a structure object, suitably converted, points to its initial member (…), and vice versa.”) but that is sufficient for our example struct User. However, if you only permit incrementing through the remaining bytes of the object then container_of() is disallowed.)

                                              Hence the <string.h> preamble is vague about which object’s end is the end that matters, and implies that it isn’t necessarily the end of the char array: “the end of an object” could just as well be the end of the enclosing struct User.

                                              The upshot is that C doesn’t allow precise bounds checking on strings embedded in structs, and a buffer overrun that corrupts fields following the string is not UB (tho it might lead to UB when those fields are subsequently used). And container_of() relies on the same kind of permitted but unsafe operation (using pointer arithmetic to access outside the bounds of a subobject) that leads to buffer overrun security vulnerabilities.

                                              Tangentially, it’s maybe worth noting that the meaning of the [ ] operator is changing in the next C standard so that it accesses arrays directly, without first converting them to pointers. So user->name[9] will be unambiguously out of bounds, whereas at the moment it’s arguably permitted. (But in practice compilers are likely to complain, because features like FORTIFY are stricter than the standard requires. So even in current C name[9] probably needs the same kind of casting to hell and back that would make it valid in future C.)

                                              1. 3

                                                The upshot is that C doesn’t allow precise bounds checking on strings embedded in structs, and a buffer overrun that corrupts fields following the string is not UB (tho it might lead to UB when those fields are subsequently used). And container_of() relies on the same kind of permitted but unsafe operation (using pointer arithmetic to access outside the bounds of a subobject) that leads to buffer overrun security vulnerabilities.

                                                I disagree. I think this is UB in standard C. GCC or clang would of course be free to explicitly define this behavior, but AFAIK none of them have documentation saying that they do so.

                                                You can use char* accesses to access objects of arbitrary type, but that does not mean that every object is a character array, just that char* can bypass the "type" part of C's typed object model. The standard very explicitly says that one can do arithmetic on pointers to access other elements of an array (6.5.6.9 in C23); by exclusion, this implies that you cannot do anything else with pointer arithmetic. If your argument were true, this part of the standard talking about pointer arithmetic in arrays would be redundant, so I think you are misreading the standard.

                                                Of course the real travesty here is that the C standard is written in a language that leaves such questions ambiguous.

                                                1. 1

                                                  I think that your reasoning is contradicted by this sentence from 6.5.1 Expressions - General (emphasis added by me):

                                                  If a value is copied into an object having no declared type using memcpy or memmove, or is copied as an array of character type, then the effective type of the modified object for that access and for subsequent accesses that do not modify the value is the effective type of the object from which the value is copied, if it has one.

                                                  Together with the sentence from 6.3.3.3 Conversions - Pointers that I quoted previously, I think this says it’s OK to do pointer arithmetic within an object that is being accessed as a char array, regardless of the declared type of the object. (Note that C defines “access” to mean modify as well as read.) (This also implies container_of() can be written in standard C without going via an integer.)

                                                  There’s a historical reason for my reading. I think your reading was correct for C89, but C99 is a lot more explicit about the bytewise representation of types. Surely those changes to C99 were intended to allow bytewise access (such as using memcpy() to copy objects) that was previously not strictly conforming.

                                                  Of course the real travesty here is that the C standard is written in a language that leaves such questions ambiguous.

                                                  Heh, yeah.

                                                  I think the reason this char aliasing is messy is because the wording was pushed into an awkward corner. I imagine that where 6.5.1 says

                                                  An object shall have its stored value accessed only by an lvalue expression that has one of the following types:

                                                  — … sensible things …

                                                  — a character type.

                                                  I guess they started with “character array”, but arrays are not modifiable lvalues (6.3.2.1 Lvalues, arrays, and function designators) so they can’t be used for all kinds of access, so “character array” would be the wrong wording in this list. Hence I suppose the wording was changed to refer to an element of a character array, without them clearly saying outright anywhere else that an object can be treated as a character array (except by unclear implication). (And without clearly saying anywhere what are the bounds on a character pointer that is obtained by conversion from a pointer to some subobject.)

                                                  1. 1

                                                    I think this says it’s OK to do pointer arithmetic within an object that is being accessed as a char array, regardless of the declared type of the object.

                                                    I don't see how that helps for container_of, where you are not accessing the object as a char array.

                                                    So I agree that char* can be used to bypass the strict aliasing requirements, but I don't see how that helps container_of.

                                                    1. 1

                                                      It depends on how container_of() is implemented, of course: you might go via intptr_t (but CHERI will be sad), or (like Linux) do arithmetic on a void pointer (using GNU C), or do arithmetic on a char pointer (standard C) which is necessarily treating the object as a char array.

                                                      The “access” part (actually dereferencing the char pointer) isn’t essential: this discussion started off by wondering to what degree it’s possible to do strict bounds checking on subobjects in C (at least when doing arithmetic on char pointers). Since then we’ve been trying to work out where is the wording in the standard that allows the kind of pointer arithmetic that is essential for it to be possible to copy an object as an array of character type. (Because if it’s possible then the struct User name buffer overflow earlier in this thread is not UB, and Fil-C’s checks are as tight as you can get without extra FORTIFY-style annotations.)

                                                      Like, is “successive increments yield pointers to the remaining bytes of the object” supposed to be an example of a more general facility in use, or a restriction on what pointer arithmetic is permitted? How is that consistent with the array-only specification of pointer arithmetic? And what are the bounds on the character array that determine whether the pointer arithmetic is defined or not? I think if the pointer arithmetic is valid then the access is necessarily valid (unless it’s one past the end, of course!) which is why the access isn’t essential, hence the contrast with container_of() helps to highlight the question of the extent of the bounds.

                                                      (And incidentally what is the point of offsetof() if not to support jumping around a structure like container_of()?)

                                                      1. 2

                                                        If what we are debating is "does Fil-C catch everything that is UB in standard C", then IMO the answer is "obviously not". I have not yet heard of Fil-C catching any of the UB around restrict, for instance.

                                                        But even outside restrict, I think this code has UB:

                                                        #include <stdio.h>
                                                        #include <assert.h>
                                                        
                                                        struct S {
                                                            int x;
                                                            int y;
                                                        };
                                                        
                                                        int main() {
                                                            struct S s;
                                                            s.x = 0;
                                                            s.y = 10;
                                                            int *xptr = &s.x;
                                                            int *yptr = xptr + 1;
                                                            assert(yptr == &s.y);
                                                            printf("%d\n", *yptr); // UB! We did ptr arithmetic outside of an array.
                                                            return 0;
                                                        }
                                                        

                                                        I could be convinced that you can use char* to access surrounding parts of the object, but I don't see a way to interpret the standard that would make the above program well-defined.

                                                        I haven't found an online playground for running Fil-C so I can't easily probe it to compare it with my understanding of UB in C, or I'd suggest more daring examples. But from the discussion here it sounds very much like Fil-C will not complain about the above program.

                                                        From my admittedly rather cursory looks at it so far, Fil-C is a UB-free standards-compliant C implementation (which is very impressive), but that does not mean that Fil-C can catch all UB -- in many cases it is UB-free by defining the behavior, which is totally standards-compliant but also very different from e.g. Miri which aims to catch all UB in Rust.

                                                        (And incidentally what is the point of offsetof() if not to support jumping around a structure like container_of()?)

                                                        That would be a good question to ask the committee. :)

                                                        1. 1

                                                          Since then we’ve been trying to work out where is the wording in the standard that allows the kind of pointer arithmetic that is essential for it to be possible to copy an object as an array of character type. (Because if it’s possible then the struct User name buffer overflow earlier in this thread is not UB,

                                                          That doesn't follow. The parts of the standard that allow you to copy an object "as" an array of character type are 6.3.2.3 (a pointer to x for any x can be converted to a pointer to char, and you can increment the pointer up to the size of the object to access each each byte of the object) and 6.5 (strict aliasing rule exception for accessing any object via an lvalue of char type). That's all you need to copy an object as an array of character type, but even together they do not imply that converting a subobject pointer to a char * lets you walk outside the bounds of the subobject, or that if strcpy is passed a pointer to a subobject that it can walk past the bounds of that subobject (it explicitly can't).

                                                          Even if it's intended to be implied that each object is overlapped by an implicit char array - and that's already not really certain - it doesn't mean that converting a pointer to a subobject into a char * yields a pointer into the middle of the containing object's associated char[]. Rather it may point at the subobject's own associated char array, which overlaps the subobject and partially overlaps the containing object (and associated char array). Yes, these arrays must have addresses in common, but since this logic already relies on the existence of overlapping objects, it doesn't mean that a pointer to one is also a valid pointer into the other, or at least it doesn't allow bypassing the 'you cannot walk a pointer past the end of an array' restriction which is explicit in 6.5.6 - I.e. if you have a pointer into a char[] that covers only the subobject, then going outside the bounds of that array is UB.

                                                          1. 1

                                                            Yeah I think your arguments (and Ralf’s) are reasonable, but they lead to weird situations, eg,

                                                            // pointer to the first element of name
                                                            user->name
                                                            // pointer to struct User
                                                            (struct User *)user->name
                                                            // pointer to first element of first member of struct User
                                                            (char *)(struct User *)user->name
                                                            // or struct User viewed as a char array
                                                            (char *)(struct User *)user->name
                                                            

                                                            The compiler can’t tell the difference in intent between the third and fourth expressions, and if the third expression is a no-op, why is the fourth expression not a no-op? Or, to be strictly correct, should the third cast look like this?

                                                            (char [8])(struct User *)user->name
                                                            

                                                            But then if a pointer to the first element of user->name is not the same as a pointer to user->name, why is the second cast OK?

                                                            Basically, I think your strict reading means that much of C As She Is Spoke does not conform to the standard: eg, it means that it isn’t valid to short-cut cast directly to the target type (if the standard doesn’t explicitly say it’s allowed), and instead you have to write casts via all the intermediate types (being careful to distinguish between arrays and pointers) so that the compiler understands you have laundered the pointer to the satisfaction of the language lawyers.

                                                            1. 1

                                                              The compiler can’t tell the difference in intent between the third and fourth expressions, and if the third expression is a no-op, why is the fourth expression not a no-op? Or, to be strictly correct, should the third cast look like this?

                                                              I would say the third isn't a no-op - you're getting a char * that can navigate the entire User object. However:

                                                              But then if a pointer to the first element of user->name is not the same as a pointer to user->name, why is the second cast OK?

                                                              It's a good observation that it's not explicitly allowed. I always assumed that that the ability to cast from an "initial member" to the container was intended to be extended to the "initial element of the initial member" (or rather, that it's transitive, and initial element works just as well as initial member, and so you can skip the intermediary cast). Glancing through now however I find no allowance for casting from an initial element pointer to a pointer to the containing array at all - an array decays to an element pointer in most contexts but that's not technically a conversion and is specified only in one direction.

                                                              Definitely the C standard is not well written and has a lot of unnecessary ambiguity. I tend to prefer conservative interpretation (I'd rather not risk my programs stop working at some point in the future when the compiler starts recognising additional UB for purposes of optimisation), but you are certainly right that a lot of extant programs require a looser reading.

                                                  2. 1

                                                    It doesn’t say that they have to be declared as char arrays,

                                                    Alright, I misread that detail. But it does say clearly that if "If an array is accessed beyond the end of an object, the behavior is undefined", which is what is relevant here.

                                                    But incrementing a pointer is only defined for arrays (6.5.7 Additive operators)

                                                    Other than how it is defined for the case described in 6.3.3.3.

                                                    if you only permit incrementing through the remaining bytes of the object then container_of() is disallowed

                                                    A container_of implementation can cast the pointer to a an integer type before subtracting the member offset and cast it back to a pointer type to get the container, so there is no incrementing or decrementing of pointers. The cast between pointer and integer is implementation-defined, so container-of is technically dependent on the implementation.

                                                    Hence the <string.h> preamble is vague about which object’s end is the end that matters

                                                    That's a non-sequitur. The preamble says "an array" which without qualification means the same thing as "any array" (likewise with "an object"). It's standardese; it's written with pedantic language and meant to be read pedantically.

                                      3. 9

                                        No, you cannot legally do that in the manner you seem to be describing. It is undefined behaviour to do so. In C99 (the reference I happen to have on hand) [...]

                                        It doesn't matter whether some spec/reference calls it illegal/undefined/etc, what matters is that the FilC compiler/runtime lets it happen.

                                        It's a clear OOB access in most languages, but FilC doesn't catch it. I wouldn't mind that hole (it has a valid technical reason) if FilC didn't gutsily claim "total memory safety".

                                        1. 4

                                          It doesn't matter whether some spec/reference calls it illegal/undefined/etc, what matters is that the FilC compiler/runtime lets it happen.

                                          The FilC compiler/runtime may well let it happen. But if there's a claim made that "the language allows this" (which there was) but it doesn't (which is the case), I'd say that matters.

                                          I wouldn't mind that hole (it has a valid technical reason) if FilC didn't gutsily claim "total memory safety"

                                          It remains a problem that there's no consensus agreement on what "memory safety" actually means. I agree that adding "total" when this hole is present is a little presumptuous.

                                          1. 4

                                            Yes, the language allows this. It doesn't matter what the spec says about this, because the spec also says that it's up to compilers to decide what to do here, and they all decide to allow this. The C spec is intentionally weak : when it calls something illegal/undefined/etc, that has as much effect as me not allowing swear words for my kids.

                                            1. 2

                                              Yes, the language allows this. It doesn't matter what the spec says about this

                                              The spec is what defines what the language itself allows. Compilers are, as you say, free to make further allowances.

                                              up to compilers to decide what to do here, and they all decide to allow this

                                              They do not allow out-of-bounds access in general, even if they do allow the specific case given by example.

                                2. 11

                                  Would be good to hear the venerable pizlonator's thoughts on this. Sadly he often resorts to jabs at Rust for marketing Fil-C, but actually a mixed system could make a lot of sense from a performance & safety POVs; the q is whether it's technically viable

                                  1. 13

                                    A hybrid fil-C/Rust world would yield a somewhat surprising incentive gradient: Rewrite your C in Rust not to make it safer but to make it faster. I wonder how that would pan out.

                                  2. 3

                                    I'd love to see a Fil-C FFI for the languages that I use that so often depend on C for libraries.