Last few weeks I started programming again a bit with MASM. I wanted to create some simple demo using Mode 13h but I'm struggling to create a 3d system.
I was wandering if anyone knows some good tutorial/book that would explain 3d software rendering from beginning?

I know that there are lot of resources out there but most of them focus on GDI and that's something I don't have interest in learning (I already did some 2d demos in it). I also know some stuff how graphics pipeline works from DirectX (shaders, stages...) but now I would like to learn how and why stuff works this way.

Edit:
I would prefer some newer reference that is up to date (not like Graphics Black Book from Michael Abarsh). The tutorial does not need to focus on assembly or any specific language, just to be mostly about software rendering.

Have you seen Andre Lamothe's book "Tricks of the 3D Game Programming Gurus-Advanced 3D Graphics and Rasterization"? I have not looked at it in years and don't remember the specifics, but it's software rendering from the beginning (shading, optimization techniques, etc.). Might be worth checking out.

https://www.amazon.com/Tricks-Programming-Gurus-Advanced-Graphics-Rasterization/dp/0672318350/ref=asap_bc?ie=UTF8

One book was called 

Keep in mind that GDI is just the interface used to draw on the screen, the software rasterizer is built on top of that, which means you can replace that with anything you want.  If you found good articles explaining how to create a software rasterizer, but it happens to use GDI to draw that to the screen, then just swap out the GDI for something else (mode 13h perhaps?).

On 7/26/2018 at 11:15 AM, AtomicWinter said:

Have you seen Andre Lamothe's book "Tricks of the 3D Game Programming Gurus-Advanced 3D Graphics and Rasterization"? I have not looked at it in years and don't remember the specifics, but it's software rendering from the beginning (shading, optimization techniques, etc.). Might be worth checking out.

https://www.amazon.com/Tricks-Programming-Gurus-Advanced-Graphics-Rasterization/dp/0672318350/ref=asap_bc?ie=UTF8

I have that book floating around my collection somewhere. It's good as far as the absolute basics of software rendering goes. The main problem with is now is that it's built around a positively ancient version of DirectX. I want to say DX7 but it's been around 10 years since I've looked at the book last. Edit: Nevermind, I have a different Andre "Tricks of the..." book. My bad.

You should be able to do software rendering using GDI by generating a bitmap in memory then presenting it using BitBlt or something similar. That would let you do the vast majority of your work by directly mucking about with pixels in memory.

That's a lot to cover, and the answer somewhat depends on what you already know.

First thing I'd recommend is getting a firm grasp on basics: concepts, terminology and mathematics involved in the subject. There's a lot of confusion going on, and for no good reason, as far as I can tell, because this things are, in fact, well defined and clearly explained - the trick is knowing where to look. Clear terminology helps immensely: e. g. when people talk, say, that "1/z is linear in screen-space" it really helps to know which z it is (disclosure: it is z in eye-space, same as w in clip-space, up to a sign).

So it helps to know: at least rudimentary basics of matrix algebra (you really don't need much to start); basics of homogeneous coordinates (how they are used in computer graphics); what the relevant coordinate systems and transformations are called (object-space -> eye-space -> clip-space -> NDC -> window coordinates); what's clipping and how it is defined (-w<=x,y,z<=+w for OpenGL); and, yes, why "1/w is linear in screen-space" (see e. g. http://www.lysator.liu.se/~mikaelk/doc/perspectivetexture/ ).

A decent text on computer graphics should clear that. I guess, "OpenGL Programming Guide: The Official Guide to Learning OpenGL" ("Red Book") works (the 1.1 version is free:  https://www.glprogramming.com/red/ ), as do some others. "Computer Graphics: Principles and Practice" is a pretty comprehensive textbook, which might be overkill for the task (but then again, it's probably worth reading anyway).

To reiterate: the point of the above is to get a grasp on the concepts - you do not necessarily need to read the textbook from cover to cover (though you might want to).

Also, GLSpec (e. g. https://www.khronos.org/registry/OpenGL/specs/gl/glspec21.pdf ) can be very helpful, as it contains (unlike most texts) much of the "implementator's view" of OpenGL. But, possibly not right away. Probably same for it's DirectX equivalent.

As an aside: mathematics tends to age rather gracefully, so the older texts (math-centric ones, that is) are probably ok.

Now, that it is dealt with, on to the actual topic of writing a software rasterizer.

The best explanation I know of is the one by Fabian Giesen:

https://fgiesen.wordpress.com/2011/07/06/a-trip-through-the-graphics-pipeline-2011-part-6/

https://fgiesen.wordpress.com/2011/07/10/a-trip-through-the-graphics-pipeline-2011-part-8/

https://fgiesen.wordpress.com/2013/02/08/triangle-rasterization-in-practice/

https://fgiesen.wordpress.com/2013/02/10/optimizing-the-basic-rasterizer/

This describes the half-space rasterizer, which is predominant type in modern rendering - both hardware and software.

Another popular type (which was a lot more common some time ago) is a scanline rasterizer. Chris Hecker has a series of articles on the subject. (see http://chrishecker.com/Miscellaneous_Technical_Articles )

Some additional useful links:

https://www.cs.drexel.edu/~david/Classes/Papers/comp175-06-pineda.pdf

https://web.archive.org/web/20171128164608/http://forum.devmaster.net/t/advanced-rasterization/6145

http://blog.simonrodriguez.fr/articles/18-02-2017_writing_a_small_software_renderer.html

https://www.scratchapixel.com/lessons/3d-basic-rendering/rasterization-practical-implementation/overview-rasterization-algorithm

The task of putting that on the screen is relatively trivial - you just generate a framebuffer in RAM and put blit it on screen using whatever API you have on hand (BitBlt, StretchDIBits, SDL_BlitSurface, glDrawPixels, what have you; or generate the image right in the video-memory, if we are talking 13h).

"thebennybox" has a very good tutorial for 3D software rendering:

https://www.youtube.com/channel/UCnlpv-hhcsAtEHKR2y2fW4Q

There's also a nice short tutorial on GitHub:

https://github.com/ssloy/tinyrenderer