You know who did have the right idea though? Dennis Ritchie, who proposed a fat pointer type for C all the way back in 1990. Would have made for a perfect addition to C99. Imagine how different the world might have been had the committee added that in.
We had a second chance with the release of the "C's greatest mistake" blog article from Walter Bright in 2007, essentially pushing for the same idea as Ritchie (slices/stringviews) but explained with much clearer language.
Alas, didn't make it to C11.
We're now in C23, still nothing. But we did get _Generic and VLAs! Party hard.
https://digitalmars.com/articles/C-biggest-mistake.html
And because it came up in my search and the bikeshedding discussion made me chuckle, reddit on same: https://www.reddit.com/r/C_Programming/comments/90uq7c/cs_bi...
Am curious about this esoterica, if anyone can confirm/deny:
>> Speaking of [C] arrays decaying into pointers, does anyone know why this behaviour was designed in the first place?
>> It was so that B code could be compiled as C with minimal changes. The designer felt that this would encourage people to switch from B to C. In B an array declaration actually defined a pointer and an array, with the pointer initialized to point to the array's first element.
This is my pet peeve of teamwork. We can choose solutions A, B or C. Each has upsides and downsides. We debate for two weeks, then we choose nothing.
VLA has been demoted to an optional feature in C11 (good).
IMHO the current main problem is that the C stdlib is stuck in the K&R era and the stdlib APIs haven't even been updated to the language features added in C99 (e.g. make use of struct args and return values). A range struct (ptr/size pair) in the stdlib and new or updated string functions to use such ranges would already go a long way.
C++ has the same issue (only with more chaos and bloat). They add some new good idea (like optional) but don't update the rest of the standard library to make use of it. And they can't really, without breaking backwards compatibility.
I think looking at the edition system in Rust could be useful for C and C++ to start to solve this. Something like "If this source file has this pragma in it, compile that code with a new edition". It would have to be granular, per expression really, to handle macros (which is how it works in Rust too). What would it change in C/C++? Name resolution, you could get a different set of resolvable overloads, depending on which edition is active in the caller context. Not unlike an enable_if.
The same would work in C: depending on the caller edition, expose function signatures.
Huh. Whenever I've been asked to review C code, I always looked for strncpy and always found a bug with it.
* NUL terminated strings (and now, non UTF-8 encoded strings on input/output)
* Using LF or CR or CRLF as line terminators, and pipe/comma-delimited fields when there were other unambiguous ASCII characters that could have been used (eg, GS, FS, RS) that would have made the encoding/decoding of line termination an I/O thing keeping HT/VT/CR/LF/FF as literally print related codes.
I suppose you could document that it's unsupported, and just drop or reject such values, but then the system couldn't be used to handle test data for such systems, for example.
Otherwise you need to have some sort of escape mechanism, exactly like quoting strings in CSV. In fact, there's an ASCII code "ESC" for entirely that purpose. :)
The problem is that those characters are non-printable, which means if you're just dumping the file out somewhere, you can't see them.
NL Next line (from EBCDIC?)
LS Line separator (invented by Unicode)
PS Paragraph separator (same)
The Unicode standard says that in addition to CR, LF, CRLF and the above, vertical tabs and form feeds should also be treated as line separators.
UTF-8 on stdin/stdout works perfectly fine (unless you are on Windows of course, which is stuck in in the early 90s when it comes to international text encoding).
> Using LF or CR or CRLF as line terminators
This is also an operating system convention, and it would be better if programming languages wouldn't try to "guess" the correct line endings, since this causes more problems than it solves - but again, this is mostly a Windows specific problem, and it's Microsoft's job to finally bring Windows into the current century.
Unix used LF, Apple used CR, Microsoft used CRLF.
They are all ASCII carriage movement codes, which is about driving the paper feed and print head of an ASR-33 or equivalent.
So they all made the "wrong" decision about what to store in a file.
They just chose different wrong characters.
Unix followed Multics. Multics chose right. ASCII/EMCA-6/ISO646 drafts discussed this at least as early as 1963¹: “For equipment which uses a single combination (called New Line) [...] NL will be coded at FE₂ [Field Effector 2 = 0x0A].”
¹ doi/10.1093/comjnl/7.3.197
I'd say Multics/Unix was technically correct, except this was still the wrong decision for I/O ever since.
The Record Separator is the logical character code to use to indicate the end of a line of text and print position characters, assuming that a line of text is a "record".
Apple hasn't been using CR since the release of OSX (26 years ago). Microsoft could have made the switch at any time too (just as they could have switched to UTF-8 as universal text encoding on Windows), they just choose not to.
In the end it's not the job of programming languages to clean up Microsoft's mess ;)
Why is it Microsoft's fault? They just stayed on their legacy implementation, Linux and Apple chose to move from the legacy implementation to another legacy implementation. That seems dumb.
Some believe Gary Kildall picked CRLF for CP/M since he used DEC TOPS-10 to develop CP/M. see https://www.quora.com/Why-did-CP-M-stick-with-the-CR-LF-stan...
The fact that both Apple's and CP/M codes came out roughly at the same time, both on microcomputers, shows that it was probably just a design decision.
Last time I had to handle CSV files in bash, I converted them internally to RS and FS.
CR is the only wrong choice. There's never a reason to go to start of line without erasing the line or moving to next line in a file. And even user interfaces will have smarter ways to do that. It's a completely useless concept outside of typewriters.
Well, CRLF (or worse, LFCR) is also obviously a wrong choice because it's pointless to demand two characters and create problems when one of them is missing when one totally unambiguous character will do.
I would just use UTF-8 everywhere.
What a nightmare, does it have to be so convoluted?
Getting strncpy right always has been
I'm sure these are the sorts of things that will go down as folklore from the "founding ages", when everyone will have forgotten how to understand source code in 50 years and the Claude/Codex cruft just silently keeps piling on and burning the majority of our planets energy.
I don't think this will happen. Human desire to understand how things work will still be around in 50 years.
And sentinel value terminations make a lot of sense when you have punch cards and fixed length records that you need to carve into pieces.
Nobody expected any decisions they were making in the 1960s and 1970s to have any bearing on computing a half-century later. They all expected to have their mistakes long papered over by smarter people at some point.
But we ALL make the mistake of underestimating inertia.
Strictly speaking, it's not alignment compatible from CString to BSTR unless you declare all strings to be at most 255 characters or the cpu architecture doesn't require aligned access for multi-byte words (like x86). The BSTR alignment must match the alignment of the length word, meaning you can't convert a randomly-aligned C string to BSTR by simply attaching a prefix in-place.
Also, having the length embedded in the value rather than in the pointer makes it impossible to create BSTR (sub)slices without performing a memcpy. Fat pointers do not have this restriction.
Yes, there is a trade-off between slices using the same format and having compatibility with C strings. Hence “middle ground”.
You can still use a string-slice type on top of BSTR, it just would be a separate additional type. Note that languages like Java also don’t have a singular type for strings and string slices.
It makes hybrids like this very dangerous for anything even remotely security-adjacent, such as roles, tokens, etc.
This kind of thing caused the CVE-2009-2408 and CVE-2009-2510 "Null Truncation in X.509 Common Name Vulnerability."
In Free Pascal for instance, strings are pointers to the first character with a header in a "negative address" containing information about the length, reference count (strings are reference counted and use copy-on-write to avoid passing around copies all the time) and codepage (FP can convert strings between different encodings "transparently").
For a while, 16bit would probably have seemed too extravagant. Now 32bit would probably seem too small.
For a “strongly typed” language, C is pretty damn loose where would have mattered.
It has a big advantage over the Pascal approach in that you can do zero-copy slicing, since the length is separate from the actual data.
And `size_t` makes perfect sense for the length here. If your strings are longer than the address space (which `size_t` technically isn't, but is practically very strongly correlated to it), then you're going to have a problem regardless of the number of bits for the length anyway.
But one would note that in order to gain memory for this particular case of slicing, one introduces 2 extra words (size and pointer) for every other cases. Like perhaps the second most common string operation, concatenation. In those other cases, the benefit is slightly negative.
I've had extensive experience with "counted strings" because I implemented a bunch of Forth interpreters which also uses this scheme. Including the common trick of using counted and zero-terminated strings, which is the worst of both worlds in the end. Forth is the kind of language that quickly show you how bad your choices are.
I eventually dropped all that and adopted ASCIIZ strings because they are generally more efficient (if you pay attention to the strlen() performance pitfalls) and having a dead simple interface with the rest of the world (OS, libraries) is more valuable.
It turns out that the machine is much better at the sort of boring mechanical tasks where thoroughness counts and imagination doesn't and so languages which do more, and more, and more checking pay off very well. Rust's borrowck is the obvious first thought today but say WUFFS will check that you've proved certain key properties, WUFFS doesn't need to insert runtime bounds checks for example because you've proved, before the code would compile, that you don't have any bounds misses. You might have proved it by writing bounds checks yourself of course, or likely you have an inherent mathematical rationale for why your algorithm has no misses, but either way the compiler checked your work.
Bounds checks and sized arrays and strings are mechanically very easy to perform by a machine. These are highly automated tasks.
There are some extreme cases where they ruin performance, but in the vast majority of cases they don't matter.
If you look at the type of tasks that cannot be automated, if going from no to full automation required an efficiency loss of 5%, most people would see taking the hit as an obvious choice.
And this is where the problem becomes recursive. You can build a language where the runtime check becomes a compile time check.
We ought to abandon the C paradigm of shifting all the work to the developer and shift more work to the machine.
Linux already takes several of these "efficiency loss" choices in C. The insistence that "actually I never make this mistake" has to be the surest sign that you're not talking to a real engineer across our whole industry. I associate it most with Bjarne Stroustrup, a man who has written a lot of books and papers but no notable software since his "cfront" C++ transpiler decades ago.
And besides all that, WUFFS isn't even taking an "efficiency loss" - remember it isn't emitting bounds checks it just checks that you proved you don't have bounds misses.
I haven’t programmed anything Pascal related for 30+ years but I dimly remember thinking at the time that I wished the string system wasn’t so hard to use.
* https://news.ycombinator.com/item?id=48614913
A C string is one pointer reaching all of memory, a Pascal string is two pointers reaching all of memory
Some implementations use more bytes for the length data, such as Delphi which changed over to a 4 byte prefix length, though those aren't technically Pascal strings anymore. I can't find anything about a Pascal string being two pointers?
In C++23, variant<> permits to do what Rust's typed enums introduced (e.g. Result sum type that is either a "real" result - with result type - or an error - with error type -, each strongly typed).
If you do that, a definition like
class IString { /* basic string functions */ };
class MiniString : public IString {};
class CZeroTerminatedString : public IString {};
class PascalString : public IString {};
class CppString : public IString {};
use String = std::variant<MiniString, CZeroTerminatedString, PascalString, CppString>; // define one type for all impl.
The developer can then pick the most suitable implementation, i.e. CMiniString for very, very short strings (that fit into 64 bits, so approx. <= 8 UTF-8 characters), CZeroTerminatedString (for char *co = "test\n"; zero-terminated old style C strings), CPascalStrings for strings that carry a length in s[0] or as a struct member or class field, and CppString as a wrapper for the C++ std::string that implements IString.
Sum types are a type-safe and memory-preserving way to do what in the older days was sometimes implemented using a "union {}" (which was not type-safe).
AFAIK a "Pascal string" is basically another way to say "length-prefixed string" (as opposed to null terminated string) and Free Pascal (and Delphi) are like that (and they're Pascal dialects too, so their strings are literally Pascal strings :-P).
FWIW all Pascal dialects since the 90s have a string type that allows more than 255 bytes. In Free Pascal strings are pointers to the first character with a header in a negative offset indicating the length, reference count and codepage (these fields are aligned depending on the CPU). For C compatibility the string is also null terminated so you can pass such a string to a C function and it'll work as expected. AFAIK Delphi also does the same.
Easy to say in hindsight.
It was an optimisation made back when every single byte mattered because you might have some kilobytes of memory and a 6502 CPU (where you strongly avoided using 16 bit pointers or arithmetic - because your program would be too bloated otherwise).
At the time Pascal was used, a whole byte for each string was seen as a waste - so fixed length strings were often used instead.
But at the same time, I think blaming the software was kind of a cop out. Devs were in a hurry and simply didnt respect the rules. Given todays software engineer at large. Nerfing programming languages so they cant destroy things might not be a bad idea. But AI will nerf everything.
Why do you assume that AI is gonna nerf everything?
See, AI was trained on existing data - on all that existing C code out there (sure, and also on all the papers and articles saying what was wrong with that C code). Those bugs are in the training data, and often not marked as bugs. So when AI generates C code, is it going to avoid making the mistakes that human code made? No, it's going to generate the kind of code it was trained on. How could it be otherwise?
That's not going to nerf anything.
The generalization of this is why I think all these AI companies writing blog posts where the marketing department is just jer—ranting endlessly about how AI will improve itself into the singularity is just crazy talk. They generate a random statistically likely output, and the most statistically likely output is mid. Exceptional outputs — the ones that wow us or move the needle are exactly that, unlikely. AGI is sci-fi, and LLMs will not change that.
You can see the same effect when AI emits bash, too, and especially so since most bash is terrible, and most users of bash do not put in the effort to learn bash and its foibles. So it outputs what most people write, which is not great.
Serious question. Anyone else seen this happen in the last 12-18 months? If so, which model and version were you using?
No modern LLM has found any buffer overflow bugs in parts of my code that originated from another LLM. Again, though, they have found one or two that were my fault.
If not, you're headed for a bad time, a major attitude adjustment, or (most likely) both.
The subdivision issue is a good perspective, but i would argue the performance impact of cloning substrings is dwarfed by the redundant full string reads to find length.
To hold the length of a string, I'd do something similar to unicode:
7-bits for size + 1-bit for continuation, then 15 bits for size + 1 bit for continuation, then 23-bits for size + 1 bit for continuation, etc.
Or maybe even do it exactly the same as unicode:
0XXX XXXX -> length of string is in those 7 bits
1XXX XXXX XXXX XXXX -> length of string is in those 7+8 bits
11XX XXXX XXXX XXXX XXXX XXXX-> length of string is in those 6+8+8 bits
...
A few more clock cycles compared to NULL-termination, although my alternatives above require even more clock cycles.
If the hardware had instructions for sentinel values, things would be easier (Like how DOS calls used '$' termination for strings) and safer.
Load a sentinel byte into a register and have dedicated copy and compare instructions that take each two addresses (src and dst) and copies (or compares) src/dst until the terminator is reached (with copy copying the sentinel as well).
Considering that sentinel values are needed so often, and are so useful, it's surprising that this is not in any ISA. What we have now is kludgy workarounds in the HLL for this. It's hard to blame the HLL, because some workaround has to be implemented.
while (*d++ = *s++)
;A zero is a sentinel value and is catered to by all ISAs.
Why would using a "$" be any easier/safer than a NUL?
> Why would using a "$" be any easier/safer than a NUL?
I didn't say it had to be '$'; I specifically said that the sentinel would be loaded into a register. In that case it could be anything, including zero (for the snippet you posted), or INT_MAX if the code iterated across an array of integers, etc.
By having rep/mov variants that use sentinels, a lot of the HLL problems go away - Java, C#, Python, etc would all look very different today if the ISAs from the 80s included sentinal variant of memory instructions.
PDP-11s, 68Ks, nearly all ISAs that I know about treat zero as special.
It falls naturally out of the ALU operations.
So why would people writing assembler code use another value unless they had to?
If, OTOH, the ISA had additional variants of those instructions that allowed usage of anything as a sentinel, HLL implementations of array would never have needed a fat pointer (length + memory).
The fat pointers are much more efficient in that you don't need to scan memory to get the length or find the end to append or take slices etc.
Especially for vectors that don't have any value that can be used as a sentinel.
The limitations were brutal. Initially you could only have 255 bytes in a string. The length of a string and the size of the allocation are now separate and you may need to think about that unused memory in your design. The problem now doubles with the introduction of UTF-8. Your string size is in bytes and you need to track characters separately.
If you want to create an array of strings you either need to specify the length of all strings and accept the memory overhead or have an array of pointers to strings. If you use an array of pointers you may end up choosing to use the 'nil' value as a sentinel that means "end of list." So we're right back where we started.
--
Because someone decided to downvote this HN has limited the speed at which I can reply. This site is tragic and I'm fully done with it now. You can spread propaganda and poorly sourced zeitgeist and be among friends but if you try to have a genuine conversation about programming languages you are made to be unwelcome immediately. Screw this.
--
> No other data structure works like this.
The linked list.
> You can't mess this up in an array
C happily decomposes arrays into pointers. You can erase your length information from the type. This was an intentional decision.
> Strings are the only data structure that assume there will be a NULL at end.
Which is why almost every string API has a version that allows you to specify the maximum length. The fact that you can use a NUL doesn't mean you have to. Which is why the concept of "sentinel values" is broadly used in many types of applications you haven't considered here.
Indeed. And the ignorance of computing history in this discussion is particularly disturbing.
The context of this particular thread is "zero terminated string is ... computing's biggest mistake". This completely ignores the situation on the ground when C was developed. At the time, people were striving for a system programming language that sat above the level of assembly but was compact enough to run within the limited resources of the then emerging mini-computer systems. The PDP-11 on which C was developed was certainly not the first mini-computer, but it was among the earliest to have a regular enough instruction set and addressing model to make a general purpose, high-level system's language possible. These systems were extremely limited in memory; the PDP-11's instruction set is limited to directly addressing at most 64KiB (code and data) and many systems of the era were hardware limited to less than that. (Indeed, I regularly run an early version of Unix, including an early C compiler, on my PDP-11/05 which is maxed out at 56KiB [of actual core]). There was no way that even a brilliant engineer like Dennis Richie was going to be able to shoe-horn in "optional" types, or the mechanics of length-value strings into a compiler that has to run in such limited space, and produce code (e.g. the Unix kernel) that has to run in even less. The fact that strings and arrays are thin abstractions on top of pointers is both a brilliant compromise in design as well as a nod to then-prevalent assembly practice. It was the exactly kind of pragmatic decision that was needed to move computing along at the time. Of course the designs from this era are antiquated now. But they were not mistakes.
A great many of those replying are many years short of having experienced anything like "the situation on the ground when C was developed". They simply have never known a day without hundreds of gigabytes or more of disk storage and 8G or more of RAM available for user processes after the OS consumes what it needs for its own work. They are "ignoring" because they simply have no basis for understanding.
while (*d++ = *s++)
;
L: MOV (R1)+, (R2)+
BNE LWhat about the UNIX and C folks propaganda of C being the first systems language, or always focusing on the original Pascal used for teaching and not everything else that followed up with Mesa, Modula-2, Ada, Object Pascal and friends, none of them with said limitations.
It was a systems programming language and the first well known/successful one.
There was BCPL and then B before that, which is why the language is called "C".
Pascal was considered a teaching language, along with "Algorithms + Data Structures = Programs" by Wirth etc.
The UCSD P-system was one of the first "IDEs" and used Pascal and a bytecode interpreter of the compiled code.
Modula-2 was barely available in the early 1980s.
Ada was mired in MIL-SPEC and expensive compilers etc.
People used FORTRAN for scientific programming, C for most everything else in the non-IBM mainframe world.
You missed quite a few between JOVIAL, and C being adopted outside Bell Labs.
Modula-2 was as widely available as C was outside UNIX and universities with access to UNIX source code.
It took a while for proper C to actually be "used for everything else", until the early 1990s actually, and by then anyone sensible would be much better with Typescript for C, aka C++.
I had a friend that tried to get everyone using Modula-2 but the "ecosystem" wasn't as great around the uni/ex-uni environments where I was.
C was pretty entrenched by the end of the 1980s, although I did use a weird embedded Pascal that was on HP-UX cross-compiling for Z80/8086 at the end of the decade, but they were the exception rather than the rule.
C++ was just a preprocessor for C and a "better C" at the time, people were still bitching about header files with function type signatures of "ANSI C" vs "good old-fashioned K&R".
We also tied onions to our belts...
Anyway on VMS most folks would be found using the VMS BASIC compiler, VMS Pascal or Bliss, until Open VMS made it yet another UNIX clone.
My first C compiler used the RatC dialect, let alone having access to a proper K&R C compiler.
By 1992, I already had access to a proper C++ compiler on MS-DOS, and C was history to me, other than scenarios were work was expected to be done in C like some university assignments, even here we were blessed with a plethora of languages between Lisp, Prolog, Smalltalk, ML, C, C++,....
And maybe it was 5/10 years earlier? Not sure. My uni days were the very early 1980s. Our university literally still made 1st years use marked sense (not even punch) cards.
I never ever liked C++, it always seemed to be tacked on to the side of C (literally at the start).
I liked the "better C" bits, but the "++" bits and the magic under the covers and then later the added layer of templates just seemed ridiculously complicated especially because we were still in the days of inheritance and "is-a" instead of "has-a" objects.
I loathed all the overloading that suddenly << meant something completely different when doing I/O and weird multiple function definitions to provide the generics.
Much preferred the Objective C idea of messages, was much more what I understood OOP to be after Smalltalk.
But by then I'd made the leap to being an "architect" and got to pontificate from on high and languages became semi-irrelevant.
You mean "C with Classes", later to be replaced by "C++" (Stroustrup pick this as favorite from a list of candidate names he crowdsourced) as implemented by Cfront.
If BCPL begat "B" and "B" begat "C", then "C" should have begatten "P".
Not sure if begatten is a word :)
Well yes, but given the number of security issues the argument is that it was in retrospect the wrong decision.
No worse than C strings then.
That isn't really a problem.
The problem with null-terminated strings is specifically what happens when you reach the end of the allocated array and there ISN'T a NULL character.
Every string function is designed to keep going until it finds the NULL character, so if a hacker gets rid of the NULL character, he can exploit pretty much any standard string manipulation function being used elsewhere in the program to manipulate whatever memory comes AFTER the string data structure.
No other data structure works like this. You can't mess this up in an array, because no function that manipulates arrays is just going to keep going until there is a null. That would be stupid because it would require users of the function to add a NULL to the end of their arrays before passing it to the function, so instead we just pass the size of the array to everything. Strings are the only data structure that assume there will be a NULL at end.
By the way, I read once that if you use UTF-32 every code point will be 4 bytes, constantly, but even then a single code point isn't necessarily a single character. Text is just complicated.
In C most data structures work like this, you keep going until you find NUL (character) or NULL (pointer). E.g. Strings, array of pointers, linked lists, etc. Of course you can add length to most of those, but it isn't the canonical/traditional way of doing things.
The null in a linked list is the null in the .next field, right? That's the way you would implemented linked lists independent of language. It's not the .value that is null.
A string is an array of characters (well, for characters representable in one byte at least) that has a specific value to represent the end of string.
It would be like if Int::MAX was reduced by 1 to make space for an Int:NUL constant that represented the end of an integer array. Or if you were creating your own ENUM, let's say for NORTH, SOUTH, EAST, WEST, and you added a fifth enumeration called Direction.NUL for use in arrays.
A little bit related: https://devblogs.microsoft.com/oldnewthing/20091008-00/?p=16...
This is patently false. Sentinel markers are used widely in array types. Consider GNU's getopt_long() function, a mainstay in GNU tools:
The argument longopts must be an array of [struct option] structures, one for each long option. Terminate the array with an element containing all zeros.
What sort of situation are you envisioning where a hacker can remove the sentinel (in the case of nul-termination) but not modify the length bytes (in the case of fat pointers)?
By contrast it's pretty unlikely for buggy code to mess the length. Add an element? +1. Remove an element? -1. Number of elements larger than capacity? Allocate a new array. Not much room for error.
No. They had trade-offs to make, and sentinel-based sequences are a needed thing, even outside of strings.
The mistake was that ISAs never looked at what HLL needed, then add the necessary instructions (I posted more about this below).
Even NULL is not a big mistake, when looked at in context of the time in which it was developed.
Part of their wonder is how they can behave differently depending on the data they’re working with. We like that feature when the data is the “good stuff” (docs, compiler messages, etc.), but how you tell that apart from “bad stuff” (prompt injection on official-seeming pages).
We basically expose LLMs to the same social-engineering vulnerabilities that humans have.
This is, and continues to be, an incredibly useful feature that makes C and C structs immensely useful concepts. Part of that does need an invalid value[1]. NULL is convenient for this and although there are some very weird JavaScript-trinity-meme-style consequences for this[2], it's such a useful concept that basically all languages that have the ability to construct pointers have a null pointer[3].
The alternative world looks like everyone inventing their own invalid values. Invalid, non-null, pointers are typically MUCH worse than null pointers for debuggability and security. If you unintentionally read/write/execute memory at 0x0 (by far the most common value for NULL), most operating systems will trap this, whereas may not necessarily if 0x12345678 is your invalid value.
[1]: Stuff like IA64 had NaT bits which were effectively an extra bit for what I assume to be this sorta thing. The problem with this is that it costs an extra bit. I don't really know much about IA64, but presumably [NaT 1] + [don't care] would be your null pointers here. I think?
[2]: Really what the standard, in my opinion, should have done is probably not make use of the null pointer UB for many different functions. A lot of compilers took the UB surrounding that to make incredibly dubious "optimizations" that broke stuff with zero actual performance benefit whatsoever
[3]: Yes, even Rust. Although some (again in my opinion) unfortunate design decisions made it so that C-Rust FFI isn't zero cost because of how it treats spans/slices
If Rust slices already make you sad, then the thing I'm cooking up will make you cry for days.
C pretends types exist with you, but once bytes hit the road, it's all real-life and segmentation faults.
Issue with types and C is while the compiler knows about them the standards committees don't want you to be able to. If C had first class types more people would abandon C++ and that can't be allowed to happen.
A similar limitation exists when peforming accesses against the cache, but at a much finer granularity.
For bytes, the alignment restriction obviously exists in the 8 bit level. You have one output byte and 64 multiplexer inputs.
If you scale this up to 8 bytes, you will need a lot of 64 Input multiplexers.
But even if you can take the silicon area hit, there is the problem of crossing cache lines and pages.
In the end, you cannot divide memory into blocks and allow primitives to cross those blocks without requesting both blocks at the same time. That's an inefficient waste of resources so why support the wasteful usecase in the first place?
To be honest, I've never seen much indication that the C and C++ committees are particularly fond of each other. They sometimes coordinate, but they're mostly content letting each other evolve in different directions. C is the way it is only after a long process of evolution away from the bits and bytes of BCPL into the strictly typed language we got from ASNI.
Though I should note that in a way, even some ISAs have one, what with e.g. separate float vs integer registers.
The historical argument and appeal to assembly is illogical here. The only real argument is that niche value optimization is too complex or too clever for the time so even if sum types were in C, nullable pointers would still exist either way.
You're telling me OCaml / Rust / Haskell compile to fairy pixie dust? Obviously their compilers figured it out and it works.
If that was the goal, it failed horribly - the gatekeeping didn't work because the popularity exploded.
> You're telling me OCaml / Rust / Haskell compile to fairy pixie dust? Obviously their compilers figured it out and it works.
I said nothing of the sort.
No, I didn't - I asked how sum types were supposed to work in an era of 64KB memory systems.
The boring cases require an enum tag in C too.
By bringing up the one thing that doesn't matter, your argument becomes purely ideological.
And which modern C compiler fits into 64KB? Even TCC needs 100KB. But that's beside the point. No machine of the last 36 (I'll push my chances, 40) years needs to fit a compiler in 64KB.
That's famously a single-pass compiler. Rust is famously unable to compile in a single pass.
It is not possible to make a borrow-checking language that compiles in a single pass.
> No machine of the last 36 (I'll push my chances, 40) years needs to fit a compiler in 64KB.
Exactly - that's why C is what it is: it wasn't a mistake, they were working under the constraints of the time. My original comment (that you appeared to disagree with) said specifically "Remember where C came from and why it was designed the way it was."
Let me ELI5 it for you: It was specifically designed to emit assembly in a single pass because of the constraints of the time.
WTF does "Hur Dur Rust Goodest!" comments mean in this context?
I probably should have replied under the other comment. I was also referring to your
> No, I didn't - I asked how sum types were supposed to work in an era of 64KB memory systems.
But context got lost between replies.
> that's why C is what it is
C famously had a big redesign in 1990. The language of today isn't the same K&R printed.
Apparently they found a way to have the CPU know... about "this stuff".
I don't know what to tell you, but you're clearly not cut out to be a software developer in either machine code, assembly or C or any other language if you don't understand something this basic.
Sum types aren't the be all end all to all issues, for example you can not representer pointer values efficiently with sum types. Even rust does not wrap up pointers with sum types. Now try to go back 37 years to C89 and ask yourself if they were going to require compilers to have stringent checks like the rust compiler does.
And, tagged unions are a thing and were a thing for a long time.
Of course it's too late to change all this today; it would have been too late even 20 years ago. But outside of f.ex. Linux kernel and some other super hardcore C libraries, a lot can be done for the world to migrate to safer constructs and away from sentinel values. And that's what languages like Rust do.
Super memory constrained environments have not been the mainstream programming work for decades and now remain limited to embedded / IoT. Not sure what the reservation against sum types is these days.
No one here is saying C is a great design, but in the context of 60 years ago, it worked out pretty well, and all the language which had additional runtime complexity (Pascal of course, but also Ada and FORTH) struggled because they didn't dial the right level of complexity
You basically end up with null/0 don’t you?
So yes, it's nulls underneath, but the developer never has to think about them.
Seeking performance we've been very prone to avoid abstractions and over and over again have shown why we need the safe abstractions.
> use the type system to help us use special values safely
... but this is not the place to explain what a type system is or what sum types/maybe/optional/etc. are.
There is a huge gap between developer expectation "it's pointing at something known" and hard reality confirmed by zillions of CVE. That's the reason optionality is prevalent in modern languages and type checkers (python, typescript), nowdays even Java has sane non-nullable types.
I wouldn't call "cause of bugs and security issues" "no cost".
> it's quite rare for a pointer in C to have the ability to be NULL
As a C programmer for more than 25 years, that is the exact opposite of my experience.
In practice really 4 because "indeterminate" is a reasonable error condition you'd like to know about.
And it keeps increasing anyway: e.g. not set has subcategories: not set due to lack of user input, not set because we're loading state from the backend etc.
NULL is the first expression of that basic problem: it's definitely not enough to eliminate NULL because the first thing which happens is your non pointer default value takes it's place.
No, a lot of pain and suffering to work around the lack of a string datatype in C.
It’s actually interesting to compare the pain and suffering of switching to a string datatype in the 80s (refactoring the limited code base then) vs the next 40 years of unnecessary boiler plate syntax and bugs for not having this type in key APIs.
But yes, the string slice type should have existed in C89 and it's very obvious from here that not having something of this sort - maybe what Rust would call &[u8] the reference to a slice of bytes - was a big problem for C.
The correct way to represent this is what's called a "fat pointer". A pair of values, one is a conventional "thin" pointer to the start of the slice, and the other is a count. Your register pressure increases in the compiler backend but problems are significantly reduced because you have fewer bounds misses.
That's a language design defect, C++ got its string slice reference (named std::string_view) only in 2017, years after Rust 1.0 shipped this as a core language feature, even though C++ is decades older.
On the other hand I can well believe on a 1970s computer where you'd be lucky to have 64kB of RAM the trade looks very different, I just think that by C89 it should have been fixed.
I guess Rust can keep slices safe using some borrow checker magic or something, but C++ can't.
If you keep const pointer to a std::string in C++ in 2016 that's exactly the same danger (of dangling pointers) as for a std::string_view in 2026, there's no change to that part.
That's not a bad thing, Common Lisp does the same and it Just Werks. The real problem is the more general "array to pointer decay", not arrays, really.
I started warning my colleagues against using it the moment I saw it for the first time about 50 years ago.
On the gripping hand, there is no strncpy in the Spinellis 7th Edition source code; 4.2BSD was using strncpy() inside readdir() in 1982, though.
https://www.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/lo...
The code, but not the function, occurred in multiple places in the V6 kernel and userland.
Yep. The code is essential given the design of the direct structure, which harkens back to the fixed-width data fields of punched cards.
"strncpy was initially introduced into the C library to deal with fixed-length name fields in structures such as directory entries. Such fields are not used in the same way as strings: the trailing null is unnecessary for a maximum-length field, and setting trailing bytes for shorter names to null assures efficient field-wise comparisons. strncpy is not by origin a "bounded strcpy," and the Committee has preferred to recognize existing practice rather than alter the function to better suit it to such use."
And I just found this comment from John Mashey (I never met John but he and I both worked under Ted Dolotta, John at Bell Labs and me at ISC in Santa Monica):
https://softwareengineering.stackexchange.com/questions/4380...
"I can answer definitively, since I wrote the originals ~1977, having moved from BTL Piscataway to Murray Hill. They were first named str*n, but were later renamed strn*, as there was some system in BTL that needed first 6 letters of external names to be unique.
I was working on kernel & user code that supported rudimentary per-process accounting, which started with someone else, but needed extensions due to big increase in UNIX systems in computer centers, who wanted more performance analysis. I.e. this was supported by commands like accton(1), acctcms(1),acctcom(1), acctmerge(1) (all in UNIX/TS 1.0, Nov 1978, which was ~Research V7 with first steps of PWB/UNIX influence. Think of that as 1.0, then PWB/UNIX 2.0, then UNIX System III...
The records described in acct(5) held the last 8 characters of the command pathname,truncated if necessary and thus possibly not null-terminated. I found multiple instances of inline code to manipulate these, which seemed a bad idea, so I wrote the str*n functions and replaced the inline code, and also used them in the various commands.
I also thought it was a good idea for better code safety.:-) Sigh."
I’m not sure why this was - the source base was so old it might have had its origins in Pascal struct behaviour.
[1]: https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
In that case, the fix is not to change C strings (breaking a lot of existing code), but to introduce a stringbuilder type.
It's a bit of extra code, yes. Not necessarily all that much, but some. On average it is only slightly more expensive than null termination, and considered as a proportion of the size of the strings themselves it's hardly anything. It's probably better than the strings getting hard-limited to 0-255, though, which was quite frequently a user-visible quirk.
The next level (110x xxxx) would give you 8MiB strings, which are going to be fine for most things.
But remember the first Macintosh shipped with 128KB of RAM, 131,072 bytes. Three more bytes per string hurts a lot more there...
... although, that said, even in that era given the number of errors that null-terminated strings caused, even completely ignoring security, I do still wonder if at least defaulting to 2 bytes of length and doing something special for strings over 64K still wouldn't have been the right tradeoff, even in the case of short strings. Today we mostly focus on security, but null-terminated strings also caused a lot of just plain-old bugs. But so did 1-byte length strings... it's way too easy to run out of 256 characters even on those old systems.
Pascal strings - historically and why people even remember this being an issue - were up to 255 chars in size, if not you had to use different string type.
You might still want raw pointers for all sorts of low level stuff, but you almost never want to have null-terminated strings for anything but back-compat, one of the worst things ever, even on memory constrained systems.
* NUL: An ASCII non-printing character with the byte value of 0
* NULL: A pointer that does not point to usable memory with the value that compiles in C to be equal to ((void *) 0).
I don’t think anyone in this thread is confusing the null character with the null pointer.
Yes it was an abbreviation in ASCII, as are all the non-printable first 32 codes.
It's just a shame that such a confusing name was chosen for such a niche use case (fixed width records that require null padding).
I was curious: Why have it, instead of just using memcpy_and_pad?
AI's answer (paraphrased) was * Avoid possible bugs from manually write sizeof(dest) * Enforces the __nonstring Attribute * signals: "I am converting an actual C-string into a fixed-width legacy memory field." vs copy binary data & pad it.
Interesting to learn about the __nonstring attribute:
https://github.com/torvalds/linux/blob/1a3746ccbb0a97bed3c06... https://github.com/search?q=repo%3Atorvalds%2Flinux+__nonstr...
Could you please elaborate on this? Both `man strncpy_s` and `man strcpy_s` didn't return any manual page on my Linux system.
For a moment, I misunderstood it as (g)libc removing strncpy and was worried about the trouble its going to cause.
The race conditions appear to be a result of the Linux kernel implementation but UNIX style syscalls introduce these races by default. It is not an inherent flaw of the API or even the implementation Linux was using.
The only useable C string API has always been memcpy anyways.
I should have sent them a nice fruit basket to commemorate the occasion.
Wouldn't this work be extremely easy to implement with an LLM coder?
Claude Code doesn't need to be interested to work.
In another comment here I explained that I have run a test: asking Claude Code to add a substantial feature to 270 different C programs.
Despite your beliefs - it went extremely well.
But xscreensaver theme tweaks for personal use have a much lower standard for quality control, regression testing, side effects, etc than a kernel used by billions of devices with thousands of interconnected drivers and subsystems.
Not to mention the coordination problem to get every maintainer on board and patches approved for each specific area when working on a project of that scale, even for a relatively narrow change.
Claude Code doesn't really help with that so don't see why the expectation would be a significant speed up (and doing it all in a single patch would definitely be rejected).
I refuse to believe the six year delay here was getting people to test a patch.
Which, actually, Claude Code will also do quite well.
Not that the Linux kernel approval procedures couldn't be streamlined, work couldn't be parallelized, or anything else like that, which would be a different discussion entirely.
You stated that Claude Code could have significantly sped up the process, so the burden of evidence here should be on how specifically these patches would have benefited/time saved from using LLMs. Hand wavingly saying "LLMs = faster" is too vague/broad of a claim without providing any evidence (and also unfalsifiable).
And what I'm saying is I refuse to believe the Linux kernel approval procedures are that inefficient. Therefore, your belief "bottleneck was most likely not mechanical code changes" is most likely incorrect.
It would be interesting to get the actual answer to this question.
EDIT: Substantially changing your argument after posting isn't nice. But to answer your charge - no - I never made that claim.
That's a different scenario, though.
Would Claude have performed adequately if it had to add a specific feature to 270 programs buried in a set of 270m program, each of which may or may not have a dependency on one or more of the others, with virtually unbounded results to test?
In terms of tokens alone, that would have been cost-prohibitive. But lets assume that you had the money to do this: it still might not even be possible.
You're confusing "I have these 270 independent programs and want to make this change to all of them" with "I have these 270m lines of code, of which only 270 needs to be changed".
Let's see if they'll let this account through.
You can find the "strncpy"s with grep, but you cannot find all the downstream effect of those changes, especially if something downstream is relying on the broken behaviour!
I took the 10 most difficult patches from the git history - the ones that took the most back-and-forth to fix. I asked Claude to write them. Would you like to see the work?
If you believe a human performs better at finding downstream effects - you need to prove that. I see no reason why it should be true.
Once gain, you are not reading what is being said - no one made that claim!
No claim was made in fact: it was a refutation. Specifically, the refutation is "this is why it took so many years".
You did not literally make that claim but your cost argument hinges on it.
Without it, then Claude does about the same as a human and only costs $100.
Apparently I'm reading your comments more thoroughly than you are.
What happens if you turn a job like that over to Claude Code? A mess? Good results? Code bloat? Worth trying on existing C programs.
It mostly did an amazing job in a short period of time.
EDIT: Of course I get downvoted for saying this. HN isn't interested in reality any more.
I suspect that rather many of us are simply just tired of Claude and friends getting shoehorned into any conversation about programming at this point. It is about as fun as the Rust Brigade entering any discussion about C. It adds nothing new to the discussion and it is frankly tiring since we pretty much at any time have a handful of conversations on the front page already covering "AI" topics anyway (counting four at the time of writing this).
I thought automation would be interesting to HN - given the context and the fact it was not used.
Pretty sure that's exactly what LLMs in coding harnesses are.
Even when they implement LLMs.
I know ChatGPT seems like it's non-deterministic - but that just a user preference.
The only real issue is if you have many people on the same server, the GPU contention can be non-deterministic in rare cases.
LLMs can be deterministic if you need them to be - it's just that most people prefer the human-like interface.
unfortunately as time goes by, the linux api surface gets larger and more convoluted. so there's going to be some coverage you're just never going to get.
but in the abstract, definitely. linux is so bloated at this point that its not clear that it can ever be 'made safe'.