why rust libraries may never exist.

rust libraries may never exist and it's for this exact reason that when you build anything in Rust you need to download like a thousand crates in cargo and spend the next decade of your life compiling your project now the rabbit hole on this one goes deep so stick around until the end I'm lowle learning I make videos on YouTube about programming and software security Now rust is one of my favorite programming languages I'm a security researcher and I honestly believe that the features in Rust like the borrow Checker and the runtime access checks will Usher

in a new era of safer software every major system in the world is written in the C language that's the Linux kernel and a bunch of other really foundational software and while C is extremely fast the language offers programmers too many opportunities to shoot themselves in the face from a security standpoint and despite the dozens of skill issue arguments and the dozens of security features that seem to get added to the C language and the C standard libraries on a daily basis still remains that 70% of security vulnerabilities originate from memory issues in C but

at the same time I completely acknowledge that rust isn't the most straightforward language to learn with the syntax is a little messy and sometimes the compiler gets mad at you and then the compile time takes forever so why is that why does rust take forever to compile and why are the binary so huge now fortunately libraries would fix all of this but unfortunately rust libraries don't actually exist and they may never exist and so before you tell me about cargo and how cargo crate are libraries you download and hold your horses cargo is a package

manager and it does manage libraries but these libraries aren't the same as libraries and languages like C let me explain every program you write in C for example depends on the gnu C library or libc functions that you use in every piece of code like open or read or write or close are already already written for you luckily and more importantly they're already compiled for you lipy exists as a shared object which lives on your file system as a file that your loader can reach into for functions that it needs to run the program at

runtime and the reason the C language is able to do this and specifically in Linux is because of the definition of the E.L application binary interface or the ABI defined for the elf file format that adheres to function calls in a way that are compatible with C this ABI exposes an interface for your program an elf to reach into another elf and find any function that it needs the elf ABI specifies a table of functions or a symbol table that a program can parse specifically to find a function that is exported by that program and

this is exactly how libc exposes function calls this ABI also guarantees that the data defined in one program is in the same order and in the same location as another so so struct x with elements a b and c will always be in that order a b and c this allows interoperability between not just functions but data in libraries as well now this is where it starts to get a little crazy rust does not at this time have a stable ABI or an application binary interface to share information across multiple binaries so cargo packages exist

but rust libraries don't all the cargo package is is a blob of source that you locally compile and it can binds all of that code into one big blob of code inside of a singular elf so effectively anytime you compile a rust program you are compiling every cargo package together required for that project and smashing them all into a single binary hence the high compile times and hence the high binary sizes so how do we fix this the answer is not simple organizing an ABI for a language and spec like the C elf is pretty

simple mainly because C is just a high level abstraction around a assembly so there isn't too much information to hide just basic types and function calls rust on the other hand is a different beast stru and rust really aren't guaranteed to be in any particular order across program boundaries as long as a and b are in the structure it's fine also generics create a whole other world of problems because they're statically dispatched and built at compile time as well as many other complex issues with types in the language also a ton of power and rust

comes from compile time static analyzers that run checks like the borrow Checker which isn't possible if there's a compiled binary if I pass a mutable reference into a compiled binary how is it possible for the borrow Checker to make sure that the reference is used in a way that is safe now I know what a lot of you are probably thinking wait doesn't rust have a cabi explicitly for this reason so yes the rust language does allow you to create types and functions that are exposed using the elf ABI using the rep C syntax reer

C decorator basically tells the r compiler to do what C does order the structure the way that c would create symbols the way the C would or the way the elf would and create a foreign function interface to call through so doesn't that kind of solve our problem no not really using reer C doesn't allow any of the rust features to cross the application boundary we can't expose a function with unique types ex using the Exotic typing system that rust has and also any function call that crosses the repr C foreign function interface is unsafe

and disables the borrow Checker reper is great for converting rust code into a C library but not for creating rust to rust binary objects now luckily I'm not the only one talking about this problem there are plenty of people much smarter than me that have open merge requests into the Russ langang Master to produce their own abis now this problem will require the entire Russ Community to get on board probably with one ABI that is the most feature complete and makes the most sense before we can make meaningful progress you can go check out this

poll request I'll put the link in the description below uh by Josh triplet which exposes the feature for an experimental feature gate to create a new API called Krabby or crab API which is pretty funny motivation here is today developers building projects incorporating multiple languages or calling a library written in one language from another often have to use the cabi and that's reer C from before as the lowest common denominator for cross language function calls as a result such cross- language calls use unsafe C representations again reper C even for types that both languages understand

for example passing an instance from rust to another high level language we typically use the unsafe C carear even if both languages have safe types for counted utf8 strings and like I said before also furthermore higher level data types such as option and rust currently require translations into cabi compatible types which discourages the use of such types in Cross language interfaces again we don't have a way of communicating a generic type an option t or result te so go check out this pull request I think it's pretty interesting the entire world of abis and how

to make systems more interoperable is really interesting to me now let me know what you think down in the comments will Rust libraries ever exist will shared objects rule the world and will Rust always compile into these gigantic amorphous monoliths if you like this video do me a favor hit like hit subscribe and then go check out this other video about an internet enabled I'm not going to say it but you you'll see