←back to thread

Understanding Memory Management, Part 1: C

(educatedguesswork.org)
237 points ekr____ | 4 comments | 13 Jan 25 14:16 UTC | HN request time: 0.887s | source
Show context
samsquire ◴[16 Jan 25 12:16 UTC] No.42724271[source]▶
Thanks for such a detailed article.

In my spare time working with C as a hobby I am usually in "vertical mode" which is different to how I would work (carefully) at work, which is just getting things done end-to-end as fast as possible, not careful at every step that we have no memory errors. So I am just trying to get something working end-to-end so I do not actually worry about memory management when writing C. So I let the operating system handle memory freeing. I am trying to get the algorithm working in my hobby time.

And since I wrote everything in Python or Javascript initially, I am usually porting from Python to C.

If I were using Rust, it would force me to be careful in the same way, due to the borrow checker.

I am curious: we have reference counting and we have Profile guided optimisation.

Could "reference counting" be compiled into a debug/profiled build and then detect which regions of time we free things in before or after (there is a happens before relation with dropping out of scopes that reference counting needs to run) to detect where to insert frees? (We Write timing metadata from the RC build, that encapsulates the happens before relationships)

Then we could recompile with a happens-before relation file that has correlations where things should be freed to be safe.

EDIT: Any discussion about those stack diagrams and alignment should include a link to this wikipedia page;

https://en.wikipedia.org/wiki/Data_structure_alignment

replies(4): >>42724597 #>>42724727 #>>42724802 #>>42725393 #
mgaunard ◴[16 Jan 25 13:27 UTC] No.42724802[source]▶
In C, not all objects need to be their own allocated entity (like they are in other languages). They can be stored in-line within another object, which means the lifetime of that object is necessarily constrained by that of its parent.

You could make every object its own allocated entity, but then you're losing most of the benefits of using C, which is the ability to control memory layout of objects.

replies(1): >>42729420 #
1. pjmlp ◴[16 Jan 25 19:02 UTC] No.42729420[source]▶
As any systems programming language include those that predate C by a decade, and still it doesn't allow full control without compiler extensions, if you really want full control of memory layout of objects, Assembly is the only way.
replies(1): >>42732644 #
2. gizmo686 ◴[17 Jan 25 00:19 UTC] No.42732644[source]▶
In practice C let's you control memory layout just fine. You might need to use __attribute__((packed)), which is technically non standard.

I've written hardware device drivers in pure C where you need need to peek and poke at specific bits on the memory bus. I defined a struct that matched the exact memory layout that the hardware specifies. Then cast an integer to a pointer to that struct type. At which point I could interact with the hardware by directly reading/writing fields if the struct (most of which were not even byte aligned).

It is not quite that simple, as you also have to deal with bypassing the cache, memory barriers, possibly virtual memory, finding the erreta that clarifies the originaly published register address was completely wrong. But I don't think any of that is what people mean when they say "memory layout".

replies(1): >>42734820 #
3. pjmlp ◴[17 Jan 25 06:55 UTC] No.42734820[source]▶
Now split the struct across registers in C.

You are aware that some of those casting tricks are UB, right?

replies(1): >>42735076 #
4. gizmo686 ◴[17 Jan 25 07:51 UTC] No.42735076{3}[source]▶
Casting integers to pointers in C is implementation defined, not UB. In practice compilers define these casts as the natural thing for the architecture you are compiling to. Since mainstream CPUs don't do anything fancy with pointer tagging, that means the implementation defined behave does exactly what you expect it to do (unless you forget that you have paging enabled and cannot simply point to a hardware memory address).

If you want to control register layout, then C is not going to help you, but that is not typically what is meant by "memory layout".

And if you want to control cache usage ... Some architectures do expose some black magic which you would need to go to assembly to access. But for the most part controlling cache involves understanding how the cache works, then controlling the memory layout and accesses to work well with the cache.