Creating a language - Static Libraries

Started by
5 comments, last by Iris_Technologies 5 years, 10 months ago

So, after days of headaches, I finally clarify a lot of how to call native functions of the operating system from a new programming language. The key are static libraries(.lib, on Windows, don't confuse with import libraries which have the same file extension). Dynamic libraries(.dll) won't do the work because:

1.- You could still read the RVA(Relative Virtual Address), from the exported table, but still even if you link the call to the function with such memory address on your binary, that would fail because, that RVA on dynamic libraries are the base address. To get the real address(process of converting RVA to VA(virtual address)) of such function, you need to relocate the RVA at runtime, because its address is shared across multiple binaries, so you would need to call LoadLibrary, FreeLibrary and GetProcAddress, which redirect us to the point 2.

2.- Because that functions are also SO calls, you can't call their RVA, because we will end with the problem of point 1.

So I were in a death point, until I read about Static Linking and Static Libraries. The concept of such kind of libraries is, they don't have RVA, because the VA of its exported functions are resolved at compile-time. But now, I faced another problem, finding information about that kind of file format is hard, and is opaqued by relocation on dynamic libraries, all that I read is by using Dumpbin and the command /SYMBOLS you could create a table of exported functions, but I can't make the users of my language download Visual Studio just for this tool. So, all I wanted to do is reading this by my own, but it's not so easy. First, I tried to read such files from the Dependency Walker, but seems like they don't use DOS Header or PE Header. So I ended up reading about how other languages did this kind of stuff, and I founded that they used a technique called FFI, which, could be described as some kind of Marshallization(it's just my point of view, may is incorrect), but this is not what I want just yet, first, I want to read the fixed VA of the exported functions of such static libraries, but seems like they are missing such information. Any clarification about it?

Advertisement
5 hours ago, Iris_Technologies said:

1.- You could still read the RVA(Relative Virtual Address), from the exported table, but still even if you link the call to the function with such memory address on your binary, that would fail because, that RVA on dynamic libraries are the base address. To get the real address(process of converting RVA to VA(virtual address)) of such function, you need to relocate the RVA at runtime, because its address is shared across multiple binaries, so you would need to call LoadLibrary, FreeLibrary and GetProcAddress, which redirect us to the point 2.

2.- Because that functions are also SO calls, you can't call their RVA, because we will end with the problem of point 1.

I don’t see the problem.  If your binaries are meant to run on Windows (regardless of the language that compiled them) then you typically will end up with a standard .EXE which will be loaded and set to run like any other.  This means you will necessarily be linking to the standard Windows libraries and the offsets will be fixed during run-time by Windows after loading your file.

Even if you have some other plan and you don’t want to call the Windows provided GetProcAddress(), implementing it manually is fairly easy.  Also note that the addresses of the most crucial Windows API functions are static across all executables of a platform (x86 or x64), so parsing just enough to get at those functions is even more trivial, but again I am not sure of the necessity.  If you plan to run on Windows then you will make a .EXE file and all the work is done.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

1 hour ago, L. Spiro said:

I don’t see the problem.  If your binaries are meant to run on Windows (regardless of the language that compiled them) then you typically will end up with a standard .EXE which will be loaded and set to run like any other.  This means you will necessarily be linking to the standard Windows libraries and the offsets will be fixed during run-time by Windows after loading your file.

Even if you have some other plan and you don’t want to call the Windows provided GetProcAddress(), implementing it manually is fairly easy.  Also note that the addresses of the most crucial Windows API functions are static across all executables of a platform (x86 or x64), so parsing just enough to get at those functions is even more trivial, but again I am not sure of the necessity.  If you plan to run on Windows then you will make a .EXE file and all the work is done.


L. Spiro

I thinked about the fact that adding a procedure to the import table of a .exe could be the answer. But i didn't think on it pretty well. So, you are telling me basically that, if i add the demangled function name and the base RVA, which i could extract from the .dll without needing to relocate, on the import table, when the .exe is loaded into memory, the RVA of the imported functions will be relocated by the OS loader and i don't need to do any black magic? 

You do some work when creating the .EXE, as you have to build your import and export tables, but that is all done via your own custom routines if it is your own custom language and compiler.
Once you have a .EXE your work is done.  Windows will use the offset table to write the correct function addresses into the correct locations in your code (or commonly a simple array of function addresses will be updated) based off your import table.  This is not your run-time task.  There is no black magic on your side.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

It might be easier if you use a premade linker and library archiver, while your compiler only produces object files in the standard format. Thats at least one step less you need to worry about.

4 minutes ago, wintertime said:

It might be easier if you use a premade linker and library archiver, while your compiler only produces object files in the standard format. Thats at least one step less you need to worry about.

Linkers not only work with native calls, they are also used to link functions and calls inside your code. One feature of my language needs to have access to the memory offset of the instructions, so I need to do it for myself. 

This topic is closed to new replies.

Advertisement