Advertisement

project organization - confused about static and dynamic libraries

Started by June 24, 2018 10:38 PM
5 comments, last by wqking 6 years, 1 month ago

It's been awhile since i put together a project from scratch and realized I don't actually understand enough about static and dynamic libraries.  I'm having a bit of trouble sussing out the confusion through google so I was hoping someone here could sort me out.

My game logic is broken up into 3 libraries:

  • GameCommon
  • GameClient (references GameCommon and implements client versions of classes)
  • GameServer (references GameCommon and implements server versions of classes)

Since the server doesn't need to do any rendering, I'm breaking up my engine into two libraries:

  • Engine
  • EngineRender (references Engine)

So theoretically my game client executable will leverage the GameClient, GameCommon, Engine, and EngineRender libraries.

Here are some of the questions I'm struggling with in regards to building that game client executable:

  1. When compiling the above libraries as static, I thought the linking step would happen once at the very end when building the actual executable.  As a result I set up my executable project to link in the 4 static libs it needed.  I was surprised to see unresolved external errors reported from the GameClient library project when using functionality from the Engine library.  I hadn't set up the GameClient library to explicitly reference (or link?) against the Engine library based on my initial assumption - I figured the library would just carry on with it's build and any holes in functionality would surface down the road when linking the executable.  Since that initial assumption seems wrong, does that instead mean that I ought to have the GameClient library link in the Engine library somehow?  Should I then assume all of the Engine library content will be inside the GameClient .lib file, or does the game executable still need to link against both GameClient.lib and Engine.lib?
  2. As a followup to question 1, if my executable only has to link against GameClient.lib and EngineRender.lib, does that mean I'm effectively pulling in two copies of Engine.lib since both GameClient and EngineRender will have to link against Engine.lib?
  3. For some reason I thought static and dynamic libraries had some high level differences but were basically interchangeable if you weren't doing anything fancy.  Swapping my libraries above from static .lib files to dynamic .dll files started causing some unresolved external issues though.  From reading around I think this is because my GameClient library is trying to use functions from Engine library which I haven't explicitly marked up as dllexport?  I feel like I have worked on .dll libraries in various projects over the years and don't have any recollection of having to meticulously manage exporting the functionality I added, but maybe I'm mistaken and this is just a huge hole in my understanding of how .dll's work?
  4. Does swapping from static to dynamic libraries impact the answers to questions 1 and 2 above?  I loosely thought the way .dll's worked was that as the executable started it would try to find the functionality it needed across the various .dll files in the directory with it and would fail to start if it didn't find what it needed.  Based on what I was seeing in question 3, it's looking like there is still some kind of linking that happens when generating the libraries and/or executables that detects when a function used from a .dll doesn't have an implementation, but maybe I'm still misunderstanding something?

 

Thanks in advance for any help anyone can provide in clarifying this stuff in my head!! :)

-Mark

Afaik, the way it works is when compiling, if a bit of code is pulled in (e.g. by calling something in a header that is defined in a .cpp, (inlines are a different matter)), or if a bit of code you pull in then pulls in another bit of code etc, then it needs all that code linked in the executable / shared library. The compiler is usually smart, so if something isn't used at all, it won't be needed. However with a dll, often it doesn't know in advance which functions you are going to call in the dll, so it has to link everything that you could possibly call. So that is one reason why you might get unresolved externals when swapping from a static lib to shared library.

The way you enforce separation between libraries often takes some careful thought and design, so that they don't become an entangled rats nest of inter-dependencies, and this is a key aspect of software design. Trying to do this at a later stage is not likely to work as you are more likely to need to rewrite a lot of stuff (you are more likely to find everything needs to be linked to everything...).

Advertisement

Dynamic Linked Libraries (.dll or it's sister .so on Unix) are in a way different to Static Linked Libraries as there code has to be fully qualified at the time you create them, so you will need any dependcie to be linked into a dynamic lib at compile time while also have to provide any dependencie at runtime. This is because compiler will add boostrapping code to the location at which it expects the libarie to stay on runtime.

A call to .dll functions have a little performance impact (at least on Windows) because your code needs to lookup the librarie on disk (via LoadLibrary) and then caches a reference pointer to the function to want to call from your code. For you, it looks like a normal function call but under the hood queries into a cached pointer.

A static librarie contains the code you want to use during runtime and the compiler links it together into one assembly. This has an impact on performance (a little, one told me ;) ) and I just static libs for my engine framework except for those code parts that need a dynamic lib (like a bridge from C++ to C# via p/invoke)

  1. 12 hours ago, Fireborn said:

    When compiling the above libraries as static, I thought the linking step would happen once at the very end when building the actual executable.

    It depends on the particular tool chain you are using. This isn't part of c++, a linker could do it any way it wants. Newer visual studios like you having dependencies set up for each module, if I remember right, whereas some other build systems aren't so picky (perhaps they do this for you). If you think it has linked against the same thing multiple times (for the same exe / dll), it will deal with this duplication for you, compilers are clever like that (same as calling the same function from different bits of your code). Calling the same function from multiple dlls is a different matter however.

  2. As above the linker will deal with it. It is in shared libraries (dlls) you are more likely to deal with duplication problems / library mismatches. If you want 2 dlls to share code, usually they would have to do something like both link to the same 3rd dll (like msvcrt for example).

  3. DLLs have a specific list of functions that are marked as being callable from the outside world. Anything else is internal. Static libraries anything is potentially accessible. Think of a DLL as a pre-compiled executable, but just with a small list of functions you can use to call it. And a static library as a bunch of object files (compiled c++) ready for linking into a bigger program.

  4. Afaik with a DLL there's no search for 'functionality' *. There's a list of required DLLs for the .exe, if they are not there, the program fails to run. If they are there, they get loaded into memory, and the predefined list of functions can be called. There's some magic fixup for the functions but this is OS / implementation specific, it could be done a number of ways, for example just using a jump table.

This is all just general info as I understand it, I've not written one of these particular compiler / linkers. This kind of thing is not part of c++ or any language, it is down to how the operating system / build system likes to do things.

* Actually you can get query function addresses and this is useful sometimes, but it is not typical use case.

I would just use static libraries unless you plan for the user to run multiple applications at the same time which use the same DLL. Then DLLs can save them ram. When using static libraries only code that is called in your app is included in the .exe, not the whole lib. It sounds like Engine library needs two types, client and server, and they can reference shared code from an external include/lib folder and be built separately.

hello

Don't forget the other advantage of dynamic libraries:

  1. They can be loaded on demand. So assume you have DLLs for different languages, you can only load one DLL for one language at certain time, without linking to all languages statically.
  2. They can be patched. So it's possible that your game downloads and updates only several DLLs without updating the whole game.

https://www.kbasm.com -- My personal website

https://github.com/wqking/eventpp  eventpp -- C++ library for event dispatcher and callback list

https://github.com/cpgf/cpgf  cpgf library -- free C++ open source library for reflection, serialization, script binding, callbacks, and meta data for OpenGL Box2D, SFML and Irrlicht.

This topic is closed to new replies.

Advertisement