Architecting and programming a graphics engine in C++ & OpenGL

[Tommy Brett]

Hi folks,

I've been working as an AS3 programmer / animator for the last 3.5 years,
and have the strongest desire to leave the marketing industry behind and
pursue my original intended career as a games programmer, which I sidelined
due to a very coincidental series of events. To this end, I've been (slowly)
building a graphics engine using C++, OpenGL and nVidia CG which renders
very large chunks of terrain using GPU Clipmaps. The idea is that eventually
it will render planets with atmosphere and real time shadows, or die trying,
and in any case will serve as my main portfolio piece when I eventually
start applying for positions. Because I've been outside of my programming
comfort zone for so long it's been slow going, but now that I've finally got
a firm grasp on shaders and the OpenGL pipeline it's time to look to the
future.

This is where my problem lies; although there are many examples on the 'net
showing me how to animate the most beautiful normal mapped cobbled cube, or
tutorials on specific rendering / culling / lighting algorithms, I've yet to
find a resource beside Michael Abrash's Graphics Programming Black Book
(which sits on my desk, reminding me of what I should have been doing all
along) which goes into the subject of writing a graphics engine. See
although I can write perfectly passable C++, and thanks to my AS3
programming experience which has frequently required that I get my projects
right 'the first time', I'm fairly adept at planning and designing my
classes before I begin... But from a C++ standpoint, I'm somewhat in the
dark. What are the best practices for building a graphics engine in C++?
Exactly how OOP should I go? How should I handle the use of multiple shaders
on chunks of geometry? There's just this giant multitude of questions, and
although I have a knack for over engineering things, I really want to have a
better picture of what I'm aiming for before I start, or I'll end up with
the monstrosity that is my terrain prototype (it runs, but it's so, so
ugly).

Currently my idea for a graphics engine involves something like an engine
class that takes render packets of data, has some sort of ability to sort
them into an order that best facilitates speedy rendering, and changes its
state depending on the information in the packets. E.g. one packet might be
a chunk of terrain, and could include information about what shaders to use,
how to determine its potential visible set, and the type of collision
detection to use (though this deviates a little from the graphics engine
part). There might be an arbitrary number of terrain packets, followed by
some character models or buildings using a different type of culling
algorithm. My worry aside from the obvious problem of the whole idea
possibly being crap, is that it's too general. How could I possibly create
an engine that handles every possible rendering type, shouldn't I perhaps
start with the assertion that I'm building a terrain engine, and base
everything around rendering sphere like chunks of geometry with varying
levels of detail? But on the same token, I don't just want to end up with
another terrain engine prototype as seen on youtube(tm).

So I suppose what I'm looking for are book suggestions, or perhaps
open-sourced graphics engines that I could look at and learn from, or maybe
I should just try and get a position as a junior-something-programmer and
learn by doing (the trouble is there aren't any such positions available
where I live, and relocation would be difficult to justify based upon the
possibly low salary expectations of a junior position, I'd like at least
some leverage). Ideas? Suggestions? Is this even the right place to ask such
broad questions? I'm all ears.

- Tommy

[Rémi Papillié]

Hi Tommy,

While this might not answer completely such a broad question, I would
recommend you look at this site:
http://gamearchitect.net/

More specifically, some articles called 'An Anatomy of Despair'
explain the design of the author's engine. See for instance:
http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views

I would also recommend the excellent book 'Game Engine Architecture',
by Jason Gregory.

Note, however, that these resources are mainly oriented towards game
engines as whole. AFAIK, there are not many resources on the specifics
of graphics engine design and architecture but, as you said, mainly a
lot of very technical articles one some technologies and
implementations.

For what it's worth, here is the structure I currently use on my
personal engine. It works, but has not been confronted to a real prod,
so what follows is definitely not bullet-proof.

The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
I use this layer mainly for readibility purposes (e.g being able to
write texture->enableFiltering(BILINEAR) instead of two or three
cryptic OGL/DX calls), but with a bit more work, you can also use it
to make the engine cross-api (with many simple but error-prone
details, such as matrix transpositions).

On top of that, two modules are present:
- The graphics scene contains an abstract description of graphical
elements that are part of the world. No shading algorithms here, just
plain buffers augmented with various information (transforms...). The
scene handles all issues about traversal and culling. Whenever a
render is asked, a render list is built by querying the spatial
partitioning structure, and then sent to the renderer (see below).
- The renderer is just the raw implementation of shading algorithms.
It receives a render list + a frustum and draws everything, using
various techniques (e.g forward or deferred shading).

The render list basically contains packets in the form <transform,
mesh, material> (+ lights), but this is highly dependent on how the
rest is implemented.

Hope this helps! I'll also be very happy to learn about other ways of
designing graphics engines, if someone else has good resources on the
subject ;)

Rémi

2011/3/18 Tommy Brett <***@gmail.com>:
> Hi folks,
> I've been working as an AS3 programmer / animator for the last 3.5 years,
> and have the strongest desire to leave the marketing industry behind and
> pursue my original intended career as a games programmer, which I sidelined
> due to a very coincidental series of events. To this end, I've been (slowly)
> building a graphics engine using C++, OpenGL and nVidia CG which renders
> very large chunks of terrain using GPU Clipmaps. The idea is that eventually
> it will render planets with atmosphere and real time shadows, or die trying,
> and in any case will serve as my main portfolio piece when I eventually
> start applying for positions. Because I've been outside of my programming
> comfort zone for so long it's been slow going, but now that I've finally got
> a firm grasp on shaders and the OpenGL pipeline it's time to look to the
> future.
> This is where my problem lies; although there are many examples on the 'net
> showing me how to animate the most beautiful normal mapped cobbled cube, or
> tutorials on specific rendering / culling / lighting algorithms, I've yet to
> find a resource beside Michael Abrash's Graphics Programming Black Book
> (which sits on my desk, reminding me of what I should have been doing all
> along) which goes into the subject of writing a graphics engine. See
> although I can write perfectly passable C++, and thanks to my AS3
> programming experience which has frequently required that I get my projects
> right 'the first time', I'm fairly adept at planning and designing my
> classes before I begin... But from a C++ standpoint, I'm somewhat in the
> dark. What are the best practices for building a graphics engine in C++?
> Exactly how OOP should I go? How should I handle the use of multiple shaders
> on chunks of geometry? There's just this giant multitude of questions, and
> although I have a knack for over engineering things, I really want to have a
> better picture of what I'm aiming for before I start, or I'll end up with
> the monstrosity that is my terrain prototype (it runs, but it's so, so
> ugly).
> Currently my idea for a graphics engine involves something like an engine
> class that takes render packets of data, has some sort of ability to sort
> them into an order that best facilitates speedy rendering, and changes its
> state depending on the information in the packets. E.g. one packet might be
> a chunk of terrain, and could include information about what shaders to use,
> how to determine its potential visible set, and the type of collision
> detection to use (though this deviates a little from the graphics engine
> part). There might be an arbitrary number of terrain packets, followed by
> some character models or buildings using a different type of culling
> algorithm. My worry aside from the obvious problem of the whole idea
> possibly being crap, is that it's too general. How could I possibly create
> an engine that handles every possible rendering type, shouldn't I perhaps
> start with the assertion that I'm building a terrain engine, and base
> everything around rendering sphere like chunks of geometry with varying
> levels of detail? But on the same token, I don't just want to end up with
> another terrain engine prototype as seen on youtube(tm).
> So I suppose what I'm looking for are book suggestions, or perhaps
> open-sourced graphics engines that I could look at and learn from, or maybe
> I should just try and get a position as a junior-something-programmer and
> learn by doing (the trouble is there aren't any such positions available
> where I live, and relocation would be difficult to justify based upon the
> possibly low salary expectations of a junior position, I'd like at least
> some leverage). Ideas? Suggestions? Is this even the right place to ask such
> broad questions? I'm all ears.
> - Tommy
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Megan Fox]

To take it in another way, I would generally recommend no one set out to
write their own graphics engine from scratch without first using another.
Simple example - OGRE. Great jumping-off point, great community, etc.
Build up a game around it, then burrow down and start changing OGRE to suit
your needs, then burrow down further and tweak the graphics, and so on.

There's very little reason to write a graphics engine yourself, unless
you're addressing the faults of a pre-existing engine you tried and found
insufficient. Without that grounding, you're stumbling in the dark, and it
will likely take quite some time and a hundred dead ends for you to get
within spitting distance of even the bad current graphics engines. You're
also needlessly re-inventing the wheel, unless Graphics Programming is
specifically what you're aiming at - there's a lot more to games programming
than rendering, and your portfolio would need to be more than rendered
planets to break in (ie. what happens on or with those planets, and why
they're fun to walk around / fly around / whatever).

----
Megan Fox
http://www.shalinor.com/
http://www.glassbottomgames.com/


On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com>wrote:

> Hi Tommy,
>
> While this might not answer completely such a broad question, I would
> recommend you look at this site:
> http://gamearchitect.net/
>
> More specifically, some articles called 'An Anatomy of Despair'
> explain the design of the author's engine. See for instance:
> http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views
>
> I would also recommend the excellent book 'Game Engine Architecture',
> by Jason Gregory.
>
> Note, however, that these resources are mainly oriented towards game
> engines as whole. AFAIK, there are not many resources on the specifics
> of graphics engine design and architecture but, as you said, mainly a
> lot of very technical articles one some technologies and
> implementations.
>
> For what it's worth, here is the structure I currently use on my
> personal engine. It works, but has not been confronted to a real prod,
> so what follows is definitely not bullet-proof.
>
> The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
> I use this layer mainly for readibility purposes (e.g being able to
> write texture->enableFiltering(BILINEAR) instead of two or three
> cryptic OGL/DX calls), but with a bit more work, you can also use it
> to make the engine cross-api (with many simple but error-prone
> details, such as matrix transpositions).
>
> On top of that, two modules are present:
> - The graphics scene contains an abstract description of graphical
> elements that are part of the world. No shading algorithms here, just
> plain buffers augmented with various information (transforms...). The
> scene handles all issues about traversal and culling. Whenever a
> render is asked, a render list is built by querying the spatial
> partitioning structure, and then sent to the renderer (see below).
> - The renderer is just the raw implementation of shading algorithms.
> It receives a render list + a frustum and draws everything, using
> various techniques (e.g forward or deferred shading).
>
> The render list basically contains packets in the form <transform,
> mesh, material> (+ lights), but this is highly dependent on how the
> rest is implemented.
>
> Hope this helps! I'll also be very happy to learn about other ways of
> designing graphics engines, if someone else has good resources on the
> subject ;)
>
> Rémi
>
> 2011/3/18 Tommy Brett <***@gmail.com>:
> > Hi folks,
> > I've been working as an AS3 programmer / animator for the last 3.5 years,
> > and have the strongest desire to leave the marketing industry behind and
> > pursue my original intended career as a games programmer, which I
> sidelined
> > due to a very coincidental series of events. To this end, I've been
> (slowly)
> > building a graphics engine using C++, OpenGL and nVidia CG which renders
> > very large chunks of terrain using GPU Clipmaps. The idea is that
> eventually
> > it will render planets with atmosphere and real time shadows, or die
> trying,
> > and in any case will serve as my main portfolio piece when I eventually
> > start applying for positions. Because I've been outside of my programming
> > comfort zone for so long it's been slow going, but now that I've finally
> got
> > a firm grasp on shaders and the OpenGL pipeline it's time to look to the
> > future.
> > This is where my problem lies; although there are many examples on the
> 'net
> > showing me how to animate the most beautiful normal mapped cobbled cube,
> or
> > tutorials on specific rendering / culling / lighting algorithms, I've yet
> to
> > find a resource beside Michael Abrash's Graphics Programming Black Book
> > (which sits on my desk, reminding me of what I should have been doing all
> > along) which goes into the subject of writing a graphics engine. See
> > although I can write perfectly passable C++, and thanks to my AS3
> > programming experience which has frequently required that I get my
> projects
> > right 'the first time', I'm fairly adept at planning and designing my
> > classes before I begin... But from a C++ standpoint, I'm somewhat in the
> > dark. What are the best practices for building a graphics engine in C++?
> > Exactly how OOP should I go? How should I handle the use of multiple
> shaders
> > on chunks of geometry? There's just this giant multitude of questions,
> and
> > although I have a knack for over engineering things, I really want to
> have a
> > better picture of what I'm aiming for before I start, or I'll end up with
> > the monstrosity that is my terrain prototype (it runs, but it's so, so
> > ugly).
> > Currently my idea for a graphics engine involves something like an engine
> > class that takes render packets of data, has some sort of ability to sort
> > them into an order that best facilitates speedy rendering, and changes
> its
> > state depending on the information in the packets. E.g. one packet might
> be
> > a chunk of terrain, and could include information about what shaders to
> use,
> > how to determine its potential visible set, and the type of collision
> > detection to use (though this deviates a little from the graphics engine
> > part). There might be an arbitrary number of terrain packets, followed by
> > some character models or buildings using a different type of culling
> > algorithm. My worry aside from the obvious problem of the whole idea
> > possibly being crap, is that it's too general. How could I possibly
> create
> > an engine that handles every possible rendering type, shouldn't I perhaps
> > start with the assertion that I'm building a terrain engine, and base
> > everything around rendering sphere like chunks of geometry with varying
> > levels of detail? But on the same token, I don't just want to end up with
> > another terrain engine prototype as seen on youtube(tm).
> > So I suppose what I'm looking for are book suggestions, or perhaps
> > open-sourced graphics engines that I could look at and learn from, or
> maybe
> > I should just try and get a position as a junior-something-programmer and
> > learn by doing (the trouble is there aren't any such positions available
> > where I live, and relocation would be difficult to justify based upon the
> > possibly low salary expectations of a junior position, I'd like at least
> > some leverage). Ideas? Suggestions? Is this even the right place to ask
> such
> > broad questions? I'm all ears.
> > - Tommy
> > _______________________________________________
> > Sweng-Gamedev mailing list
> > Sweng-***@lists.midnightryder.com
> >
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >
> >
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>

[Nicholas Aelick]

I would tend to agree with Megan. Building a graphics engine from scratch
for the learning experience is a little like building a computer by
soldering bits of wire together for the learning experience. You're unlikely
to get far, and even if you succeed, you're unlikely to end up with
something nicer then is already available.

I would recommend looking into existing solutions (partially because any
place you work for will have you working with those, anyways). If you're
really serious about building a custom graphics engine, find one that's open
source, and see how they solved the problems you've been having.

Nicholas

On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <***@gmail.com> wrote:

> To take it in another way, I would generally recommend no one set out to
> write their own graphics engine from scratch without first using another.
> Simple example - OGRE. Great jumping-off point, great community, etc.
> Build up a game around it, then burrow down and start changing OGRE to suit
> your needs, then burrow down further and tweak the graphics, and so on.
>
> There's very little reason to write a graphics engine yourself, unless
> you're addressing the faults of a pre-existing engine you tried and found
> insufficient. Without that grounding, you're stumbling in the dark, and it
> will likely take quite some time and a hundred dead ends for you to get
> within spitting distance of even the bad current graphics engines. You're
> also needlessly re-inventing the wheel, unless Graphics Programming is
> specifically what you're aiming at - there's a lot more to games programming
> than rendering, and your portfolio would need to be more than rendered
> planets to break in (ie. what happens on or with those planets, and why
> they're fun to walk around / fly around / whatever).
>
> ----
> Megan Fox
> http://www.shalinor.com/
> http://www.glassbottomgames.com/
>
>
>
> On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com>wrote:
>
>> Hi Tommy,
>>
>> While this might not answer completely such a broad question, I would
>> recommend you look at this site:
>> http://gamearchitect.net/
>>
>> More specifically, some articles called 'An Anatomy of Despair'
>> explain the design of the author's engine. See for instance:
>> http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views
>>
>> I would also recommend the excellent book 'Game Engine Architecture',
>> by Jason Gregory.
>>
>> Note, however, that these resources are mainly oriented towards game
>> engines as whole. AFAIK, there are not many resources on the specifics
>> of graphics engine design and architecture but, as you said, mainly a
>> lot of very technical articles one some technologies and
>> implementations.
>>
>> For what it's worth, here is the structure I currently use on my
>> personal engine. It works, but has not been confronted to a real prod,
>> so what follows is definitely not bullet-proof.
>>
>> The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
>> I use this layer mainly for readibility purposes (e.g being able to
>> write texture->enableFiltering(BILINEAR) instead of two or three
>> cryptic OGL/DX calls), but with a bit more work, you can also use it
>> to make the engine cross-api (with many simple but error-prone
>> details, such as matrix transpositions).
>>
>> On top of that, two modules are present:
>> - The graphics scene contains an abstract description of graphical
>> elements that are part of the world. No shading algorithms here, just
>> plain buffers augmented with various information (transforms...). The
>> scene handles all issues about traversal and culling. Whenever a
>> render is asked, a render list is built by querying the spatial
>> partitioning structure, and then sent to the renderer (see below).
>> - The renderer is just the raw implementation of shading algorithms.
>> It receives a render list + a frustum and draws everything, using
>> various techniques (e.g forward or deferred shading).
>>
>> The render list basically contains packets in the form <transform,
>> mesh, material> (+ lights), but this is highly dependent on how the
>> rest is implemented.
>>
>> Hope this helps! I'll also be very happy to learn about other ways of
>> designing graphics engines, if someone else has good resources on the
>> subject ;)
>>
>> Rémi
>>
>> 2011/3/18 Tommy Brett <***@gmail.com>:
>> > Hi folks,
>> > I've been working as an AS3 programmer / animator for the last 3.5
>> years,
>> > and have the strongest desire to leave the marketing industry behind and
>> > pursue my original intended career as a games programmer, which I
>> sidelined
>> > due to a very coincidental series of events. To this end, I've been
>> (slowly)
>> > building a graphics engine using C++, OpenGL and nVidia CG which renders
>> > very large chunks of terrain using GPU Clipmaps. The idea is that
>> eventually
>> > it will render planets with atmosphere and real time shadows, or die
>> trying,
>> > and in any case will serve as my main portfolio piece when I eventually
>> > start applying for positions. Because I've been outside of my
>> programming
>> > comfort zone for so long it's been slow going, but now that I've finally
>> got
>> > a firm grasp on shaders and the OpenGL pipeline it's time to look to the
>> > future.
>> > This is where my problem lies; although there are many examples on the
>> 'net
>> > showing me how to animate the most beautiful normal mapped cobbled cube,
>> or
>> > tutorials on specific rendering / culling / lighting algorithms, I've
>> yet to
>> > find a resource beside Michael Abrash's Graphics Programming Black Book
>> > (which sits on my desk, reminding me of what I should have been doing
>> all
>> > along) which goes into the subject of writing a graphics engine. See
>> > although I can write perfectly passable C++, and thanks to my AS3
>> > programming experience which has frequently required that I get my
>> projects
>> > right 'the first time', I'm fairly adept at planning and designing my
>> > classes before I begin... But from a C++ standpoint, I'm somewhat in the
>> > dark. What are the best practices for building a graphics engine in C++?
>> > Exactly how OOP should I go? How should I handle the use of multiple
>> shaders
>> > on chunks of geometry? There's just this giant multitude of questions,
>> and
>> > although I have a knack for over engineering things, I really want to
>> have a
>> > better picture of what I'm aiming for before I start, or I'll end up
>> with
>> > the monstrosity that is my terrain prototype (it runs, but it's so, so
>> > ugly).
>> > Currently my idea for a graphics engine involves something like an
>> engine
>> > class that takes render packets of data, has some sort of ability to
>> sort
>> > them into an order that best facilitates speedy rendering, and changes
>> its
>> > state depending on the information in the packets. E.g. one packet might
>> be
>> > a chunk of terrain, and could include information about what shaders to
>> use,
>> > how to determine its potential visible set, and the type of collision
>> > detection to use (though this deviates a little from the graphics engine
>> > part). There might be an arbitrary number of terrain packets, followed
>> by
>> > some character models or buildings using a different type of culling
>> > algorithm. My worry aside from the obvious problem of the whole idea
>> > possibly being crap, is that it's too general. How could I possibly
>> create
>> > an engine that handles every possible rendering type, shouldn't I
>> perhaps
>> > start with the assertion that I'm building a terrain engine, and base
>> > everything around rendering sphere like chunks of geometry with varying
>> > levels of detail? But on the same token, I don't just want to end up
>> with
>> > another terrain engine prototype as seen on youtube(tm).
>> > So I suppose what I'm looking for are book suggestions, or perhaps
>> > open-sourced graphics engines that I could look at and learn from, or
>> maybe
>> > I should just try and get a position as a junior-something-programmer
>> and
>> > learn by doing (the trouble is there aren't any such positions available
>> > where I live, and relocation would be difficult to justify based upon
>> the
>> > possibly low salary expectations of a junior position, I'd like at least
>> > some leverage). Ideas? Suggestions? Is this even the right place to ask
>> such
>> > broad questions? I'm all ears.
>> > - Tommy
>> > _______________________________________________
>> > Sweng-Gamedev mailing list
>> > Sweng-***@lists.midnightryder.com
>> >
>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>> >
>> >
>> _______________________________________________
>> Sweng-Gamedev mailing list
>> Sweng-***@lists.midnightryder.com
>>
>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Oladotun Rominiyi]

I would recommend Jason Gregory's book Game Engine Architecture<http://www.gameenginebook.com/>...

Might be quite useful to help you understand how all the bits of a modern graphics engine fits together...

However I'd also Echo Megan in that you should start by taken a look at a range of current engines available, even commercial ones such as UDK & Unity as they should help you understand the route data goes through (from DCC tools, to processors, to engine) which can help you, at least infer from a higher level how such systems/pipelines could work.

Game engine architecture is all about the data after all...

Oladotun Rominiyi
Curve Studios Ltd

________________________________
From: sweng-gamedev-***@lists.midnightryder.com [mailto:sweng-gamedev-***@lists.midnightryder.com] On Behalf Of Nicholas Aelick
Sent: 18 March 2011 14:16
To: sweng-***@midnightryder.com
Subject: Re: [Sweng-Gamedev] Architecting and programming a graphics engine in C++ & OpenGL

I would tend to agree with Megan. Building a graphics engine from scratch for the learning experience is a little like building a computer by soldering bits of wire together for the learning experience. You're unlikely to get far, and even if you succeed, you're unlikely to end up with something nicer then is already available.

I would recommend looking into existing solutions (partially because any place you work for will have you working with those, anyways). If you're really serious about building a custom graphics engine, find one that's open source, and see how they solved the problems you've been having.

Nicholas
On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <***@gmail.com<mailto:***@gmail.com>> wrote:
To take it in another way, I would generally recommend no one set out to write their own graphics engine from scratch without first using another. Simple example - OGRE. Great jumping-off point, great community, etc. Build up a game around it, then burrow down and start changing OGRE to suit your needs, then burrow down further and tweak the graphics, and so on.

There's very little reason to write a graphics engine yourself, unless you're addressing the faults of a pre-existing engine you tried and found insufficient. Without that grounding, you're stumbling in the dark, and it will likely take quite some time and a hundred dead ends for you to get within spitting distance of even the bad current graphics engines. You're also needlessly re-inventing the wheel, unless Graphics Programming is specifically what you're aiming at - there's a lot more to games programming than rendering, and your portfolio would need to be more than rendered planets to break in (ie. what happens on or with those planets, and why they're fun to walk around / fly around / whatever).

----
Megan Fox
http://www.shalinor.com/
http://www.glassbottomgames.com/


On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com<mailto:***@gmail.com>> wrote:
Hi Tommy,

While this might not answer completely such a broad question, I would
recommend you look at this site:
http://gamearchitect.net/

More specifically, some articles called 'An Anatomy of Despair'
explain the design of the author's engine. See for instance:
http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views

I would also recommend the excellent book 'Game Engine Architecture',
by Jason Gregory.

Note, however, that these resources are mainly oriented towards game
engines as whole. AFAIK, there are not many resources on the specifics
of graphics engine design and architecture but, as you said, mainly a
lot of very technical articles one some technologies and
implementations.

For what it's worth, here is the structure I currently use on my
personal engine. It works, but has not been confronted to a real prod,
so what follows is definitely not bullet-proof.

The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
I use this layer mainly for readibility purposes (e.g being able to
write texture->enableFiltering(BILINEAR) instead of two or three
cryptic OGL/DX calls), but with a bit more work, you can also use it
to make the engine cross-api (with many simple but error-prone
details, such as matrix transpositions).

On top of that, two modules are present:
- The graphics scene contains an abstract description of graphical
elements that are part of the world. No shading algorithms here, just
plain buffers augmented with various information (transforms...). The
scene handles all issues about traversal and culling. Whenever a
render is asked, a render list is built by querying the spatial
partitioning structure, and then sent to the renderer (see below).
- The renderer is just the raw implementation of shading algorithms.
It receives a render list + a frustum and draws everything, using
various techniques (e.g forward or deferred shading).

The render list basically contains packets in the form <transform,
mesh, material> (+ lights), but this is highly dependent on how the
rest is implemented.

Hope this helps! I'll also be very happy to learn about other ways of
designing graphics engines, if someone else has good resources on the
subject ;)

Rémi

2011/3/18 Tommy Brett <***@gmail.com<mailto:***@gmail.com>>:
> Hi folks,
> I've been working as an AS3 programmer / animator for the last 3.5 years,
> and have the strongest desire to leave the marketing industry behind and
> pursue my original intended career as a games programmer, which I sidelined
> due to a very coincidental series of events. To this end, I've been (slowly)
> building a graphics engine using C++, OpenGL and nVidia CG which renders
> very large chunks of terrain using GPU Clipmaps. The idea is that eventually
> it will render planets with atmosphere and real time shadows, or die trying,
> and in any case will serve as my main portfolio piece when I eventually
> start applying for positions. Because I've been outside of my programming
> comfort zone for so long it's been slow going, but now that I've finally got
> a firm grasp on shaders and the OpenGL pipeline it's time to look to the
> future.
> This is where my problem lies; although there are many examples on the 'net
> showing me how to animate the most beautiful normal mapped cobbled cube, or
> tutorials on specific rendering / culling / lighting algorithms, I've yet to
> find a resource beside Michael Abrash's Graphics Programming Black Book
> (which sits on my desk, reminding me of what I should have been doing all
> along) which goes into the subject of writing a graphics engine. See
> although I can write perfectly passable C++, and thanks to my AS3
> programming experience which has frequently required that I get my projects
> right 'the first time', I'm fairly adept at planning and designing my
> classes before I begin... But from a C++ standpoint, I'm somewhat in the
> dark. What are the best practices for building a graphics engine in C++?
> Exactly how OOP should I go? How should I handle the use of multiple shaders
> on chunks of geometry? There's just this giant multitude of questions, and
> although I have a knack for over engineering things, I really want to have a
> better picture of what I'm aiming for before I start, or I'll end up with
> the monstrosity that is my terrain prototype (it runs, but it's so, so
> ugly).
> Currently my idea for a graphics engine involves something like an engine
> class that takes render packets of data, has some sort of ability to sort
> them into an order that best facilitates speedy rendering, and changes its
> state depending on the information in the packets. E.g. one packet might be
> a chunk of terrain, and could include information about what shaders to use,
> how to determine its potential visible set, and the type of collision
> detection to use (though this deviates a little from the graphics engine
> part). There might be an arbitrary number of terrain packets, followed by
> some character models or buildings using a different type of culling
> algorithm. My worry aside from the obvious problem of the whole idea
> possibly being crap, is that it's too general. How could I possibly create
> an engine that handles every possible rendering type, shouldn't I perhaps
> start with the assertion that I'm building a terrain engine, and base
> everything around rendering sphere like chunks of geometry with varying
> levels of detail? But on the same token, I don't just want to end up with
> another terrain engine prototype as seen on youtube(tm).
> So I suppose what I'm looking for are book suggestions, or perhaps
> open-sourced graphics engines that I could look at and learn from, or maybe
> I should just try and get a position as a junior-something-programmer and
> learn by doing (the trouble is there aren't any such positions available
> where I live, and relocation would be difficult to justify based upon the
> possibly low salary expectations of a junior position, I'd like at least
> some leverage). Ideas? Suggestions? Is this even the right place to ask such
> broad questions? I'm all ears.
> - Tommy
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com<mailto:Sweng-***@lists.midnightryder.com>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Peter J. B. Lewis]

Hi Tommy,



I would recommend doing both: try Megan’s suggestion of trying an existing
graphics API before diving straight into the lower levels. It’s always good
to check out how existing solutions work, but I still think you will find
value in building your own from the ground up. While Nicholas is right –
you’re never going to build one better than one that’s already available –it
will nevertheless be a great learning experience (and fun!) It also means
you’ll begin to understand the design decisions made by the other graphics
engines that you will have tried out in the meantime.



It’s the difference between being a racing driver and a mechanic, really: If
you like the high-level (writing shaders, making pretty things) stick to an
existing one. If you like the low-level (architecture, speed), roll your
own. Or do both in whichever order you want :)



Regarding your choice of programming paradigm, there’s been a recent shift
away from OOP and more towards data-oriented architectures. (Sorry to throw
in more things to confuse you at this early stage, but the power of the
internet compels me.) Rendering engines tend to deal with large subsets of
data that need to be processed very fast, so OOP tends to be
counter-intuitive. I recommend reading the Managing Coupling series of
articles on the BitSquid blog if you want to know more about the reasoning
behind this. (http://bitsquid.blogspot.com/2011/01/managing-coupling.html)



P.





From: sweng-gamedev-***@lists.midnightryder.com
[mailto:sweng-gamedev-***@lists.midnightryder.com] On Behalf Of Nicholas
Aelick
Sent: 18 March 2011 14:16
To: sweng-***@midnightryder.com
Subject: Re: [Sweng-Gamedev] Architecting and programming a graphics engine
in C++ & OpenGL



I would tend to agree with Megan. Building a graphics engine from scratch
for the learning experience is a little like building a computer by
soldering bits of wire together for the learning experience. You're unlikely
to get far, and even if you succeed, you're unlikely to end up with
something nicer then is already available.



I would recommend looking into existing solutions (partially because any
place you work for will have you working with those, anyways). If you're
really serious about building a custom graphics engine, find one that's open
source, and see how they solved the problems you've been having.



Nicholas

On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <***@gmail.com> wrote:

To take it in another way, I would generally recommend no one set out to
write their own graphics engine from scratch without first using another.
Simple example - OGRE. Great jumping-off point, great community, etc.
Build up a game around it, then burrow down and start changing OGRE to suit
your needs, then burrow down further and tweak the graphics, and so on.

There's very little reason to write a graphics engine yourself, unless
you're addressing the faults of a pre-existing engine you tried and found
insufficient. Without that grounding, you're stumbling in the dark, and it
will likely take quite some time and a hundred dead ends for you to get
within spitting distance of even the bad current graphics engines. You're
also needlessly re-inventing the wheel, unless Graphics Programming is
specifically what you're aiming at - there's a lot more to games programming
than rendering, and your portfolio would need to be more than rendered
planets to break in (ie. what happens on or with those planets, and why
they're fun to walk around / fly around / whatever).

----
Megan Fox
http://www.shalinor.com/
http://www.glassbottomgames.com/





On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com>
wrote:

Hi Tommy,

While this might not answer completely such a broad question, I would
recommend you look at this site:
http://gamearchitect.net/

More specifically, some articles called 'An Anatomy of Despair'
explain the design of the author's engine. See for instance:
http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views

I would also recommend the excellent book 'Game Engine Architecture',
by Jason Gregory.

Note, however, that these resources are mainly oriented towards game
engines as whole. AFAIK, there are not many resources on the specifics
of graphics engine design and architecture but, as you said, mainly a
lot of very technical articles one some technologies and
implementations.

For what it's worth, here is the structure I currently use on my
personal engine. It works, but has not been confronted to a real prod,
so what follows is definitely not bullet-proof.

The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
I use this layer mainly for readibility purposes (e.g being able to
write texture->enableFiltering(BILINEAR) instead of two or three
cryptic OGL/DX calls), but with a bit more work, you can also use it
to make the engine cross-api (with many simple but error-prone
details, such as matrix transpositions).

On top of that, two modules are present:
- The graphics scene contains an abstract description of graphical
elements that are part of the world. No shading algorithms here, just
plain buffers augmented with various information (transforms...). The
scene handles all issues about traversal and culling. Whenever a
render is asked, a render list is built by querying the spatial
partitioning structure, and then sent to the renderer (see below).
- The renderer is just the raw implementation of shading algorithms.
It receives a render list + a frustum and draws everything, using
various techniques (e.g forward or deferred shading).

The render list basically contains packets in the form <transform,
mesh, material> (+ lights), but this is highly dependent on how the
rest is implemented.

Hope this helps! I'll also be very happy to learn about other ways of
designing graphics engines, if someone else has good resources on the
subject ;)

Rémi

2011/3/18 Tommy Brett <***@gmail.com>:

> Hi folks,
> I've been working as an AS3 programmer / animator for the last 3.5 years,
> and have the strongest desire to leave the marketing industry behind and
> pursue my original intended career as a games programmer, which I
sidelined
> due to a very coincidental series of events. To this end, I've been
(slowly)
> building a graphics engine using C++, OpenGL and nVidia CG which renders
> very large chunks of terrain using GPU Clipmaps. The idea is that
eventually
> it will render planets with atmosphere and real time shadows, or die
trying,
> and in any case will serve as my main portfolio piece when I eventually
> start applying for positions. Because I've been outside of my programming
> comfort zone for so long it's been slow going, but now that I've finally
got
> a firm grasp on shaders and the OpenGL pipeline it's time to look to the
> future.
> This is where my problem lies; although there are many examples on the
'net
> showing me how to animate the most beautiful normal mapped cobbled cube,
or
> tutorials on specific rendering / culling / lighting algorithms, I've yet
to
> find a resource beside Michael Abrash's Graphics Programming Black Book
> (which sits on my desk, reminding me of what I should have been doing all
> along) which goes into the subject of writing a graphics engine. See
> although I can write perfectly passable C++, and thanks to my AS3
> programming experience which has frequently required that I get my
projects
> right 'the first time', I'm fairly adept at planning and designing my
> classes before I begin... But from a C++ standpoint, I'm somewhat in the
> dark. What are the best practices for building a graphics engine in C++?
> Exactly how OOP should I go? How should I handle the use of multiple
shaders
> on chunks of geometry? There's just this giant multitude of questions, and
> although I have a knack for over engineering things, I really want to have
a
> better picture of what I'm aiming for before I start, or I'll end up with
> the monstrosity that is my terrain prototype (it runs, but it's so, so
> ugly).
> Currently my idea for a graphics engine involves something like an engine
> class that takes render packets of data, has some sort of ability to sort
> them into an order that best facilitates speedy rendering, and changes its
> state depending on the information in the packets. E.g. one packet might
be
> a chunk of terrain, and could include information about what shaders to
use,
> how to determine its potential visible set, and the type of collision
> detection to use (though this deviates a little from the graphics engine
> part). There might be an arbitrary number of terrain packets, followed by
> some character models or buildings using a different type of culling
> algorithm. My worry aside from the obvious problem of the whole idea
> possibly being crap, is that it's too general. How could I possibly create
> an engine that handles every possible rendering type, shouldn't I perhaps
> start with the assertion that I'm building a terrain engine, and base
> everything around rendering sphere like chunks of geometry with varying
> levels of detail? But on the same token, I don't just want to end up with
> another terrain engine prototype as seen on youtube(tm).
> So I suppose what I'm looking for are book suggestions, or perhaps
> open-sourced graphics engines that I could look at and learn from, or
maybe
> I should just try and get a position as a junior-something-programmer and
> learn by doing (the trouble is there aren't any such positions available
> where I live, and relocation would be difficult to justify based upon the
> possibly low salary expectations of a junior position, I'd like at least
> some leverage). Ideas? Suggestions? Is this even the right place to ask
such
> broad questions? I'm all ears.
> - Tommy

> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
>
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Oladotun Rominiyi]

To add to Tommy's point regarding DOD vs OOD (since DOD can be done via OOP; an argument for another time...)

Here's some presentations from DICE on the subject:-

http://publications.dice.se/publications.asp?show_category=yes&which_category=Engineering

Good luck!



Oladotun Rominiyi
Curve Studios

________________________________
From: sweng-gamedev-***@lists.midnightryder.com [mailto:sweng-gamedev-***@lists.midnightryder.com] On Behalf Of Peter J. B. Lewis
Sent: 18 March 2011 14:34
To: sweng-***@midnightryder.com
Subject: Re: [Sweng-Gamedev] Architecting and programming a graphics engine in C++ & OpenGL

Hi Tommy,

I would recommend doing both: try Megan's suggestion of trying an existing graphics API before diving straight into the lower levels. It's always good to check out how existing solutions work, but I still think you will find value in building your own from the ground up. While Nicholas is right - you're never going to build one better than one that's already available -it will nevertheless be a great learning experience (and fun!) It also means you'll begin to understand the design decisions made by the other graphics engines that you will have tried out in the meantime.

It's the difference between being a racing driver and a mechanic, really: If you like the high-level (writing shaders, making pretty things) stick to an existing one. If you like the low-level (architecture, speed), roll your own. Or do both in whichever order you want :)

Regarding your choice of programming paradigm, there's been a recent shift away from OOP and more towards data-oriented architectures. (Sorry to throw in more things to confuse you at this early stage, but the power of the internet compels me.) Rendering engines tend to deal with large subsets of data that need to be processed very fast, so OOP tends to be counter-intuitive. I recommend reading the Managing Coupling series of articles on the BitSquid blog if you want to know more about the reasoning behind this. (http://bitsquid.blogspot.com/2011/01/managing-coupling.html)

P.


From: sweng-gamedev-***@lists.midnightryder.com [mailto:sweng-gamedev-***@lists.midnightryder.com] On Behalf Of Nicholas Aelick
Sent: 18 March 2011 14:16
To: sweng-***@midnightryder.com
Subject: Re: [Sweng-Gamedev] Architecting and programming a graphics engine in C++ & OpenGL

I would tend to agree with Megan. Building a graphics engine from scratch for the learning experience is a little like building a computer by soldering bits of wire together for the learning experience. You're unlikely to get far, and even if you succeed, you're unlikely to end up with something nicer then is already available.

I would recommend looking into existing solutions (partially because any place you work for will have you working with those, anyways). If you're really serious about building a custom graphics engine, find one that's open source, and see how they solved the problems you've been having.

Nicholas
On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <***@gmail.com<mailto:***@gmail.com>> wrote:
To take it in another way, I would generally recommend no one set out to write their own graphics engine from scratch without first using another. Simple example - OGRE. Great jumping-off point, great community, etc. Build up a game around it, then burrow down and start changing OGRE to suit your needs, then burrow down further and tweak the graphics, and so on.

There's very little reason to write a graphics engine yourself, unless you're addressing the faults of a pre-existing engine you tried and found insufficient. Without that grounding, you're stumbling in the dark, and it will likely take quite some time and a hundred dead ends for you to get within spitting distance of even the bad current graphics engines. You're also needlessly re-inventing the wheel, unless Graphics Programming is specifically what you're aiming at - there's a lot more to games programming than rendering, and your portfolio would need to be more than rendered planets to break in (ie. what happens on or with those planets, and why they're fun to walk around / fly around / whatever).

----
Megan Fox
http://www.shalinor.com/
http://www.glassbottomgames.com/

On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com<mailto:***@gmail.com>> wrote:
Hi Tommy,

While this might not answer completely such a broad question, I would
recommend you look at this site:
http://gamearchitect.net/

More specifically, some articles called 'An Anatomy of Despair'
explain the design of the author's engine. See for instance:
http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views

I would also recommend the excellent book 'Game Engine Architecture',
by Jason Gregory.

Note, however, that these resources are mainly oriented towards game
engines as whole. AFAIK, there are not many resources on the specifics
of graphics engine design and architecture but, as you said, mainly a
lot of very technical articles one some technologies and
implementations.

For what it's worth, here is the structure I currently use on my
personal engine. It works, but has not been confronted to a real prod,
so what follows is definitely not bullet-proof.

The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
I use this layer mainly for readibility purposes (e.g being able to
write texture->enableFiltering(BILINEAR) instead of two or three
cryptic OGL/DX calls), but with a bit more work, you can also use it
to make the engine cross-api (with many simple but error-prone
details, such as matrix transpositions).

On top of that, two modules are present:
- The graphics scene contains an abstract description of graphical
elements that are part of the world. No shading algorithms here, just
plain buffers augmented with various information (transforms...). The
scene handles all issues about traversal and culling. Whenever a
render is asked, a render list is built by querying the spatial
partitioning structure, and then sent to the renderer (see below).
- The renderer is just the raw implementation of shading algorithms.
It receives a render list + a frustum and draws everything, using
various techniques (e.g forward or deferred shading).

The render list basically contains packets in the form <transform,
mesh, material> (+ lights), but this is highly dependent on how the
rest is implemented.

Hope this helps! I'll also be very happy to learn about other ways of
designing graphics engines, if someone else has good resources on the
subject ;)

Rémi

2011/3/18 Tommy Brett <***@gmail.com<mailto:***@gmail.com>>:
> Hi folks,
> I've been working as an AS3 programmer / animator for the last 3.5 years,
> and have the strongest desire to leave the marketing industry behind and
> pursue my original intended career as a games programmer, which I sidelined
> due to a very coincidental series of events. To this end, I've been (slowly)
> building a graphics engine using C++, OpenGL and nVidia CG which renders
> very large chunks of terrain using GPU Clipmaps. The idea is that eventually
> it will render planets with atmosphere and real time shadows, or die trying,
> and in any case will serve as my main portfolio piece when I eventually
> start applying for positions. Because I've been outside of my programming
> comfort zone for so long it's been slow going, but now that I've finally got
> a firm grasp on shaders and the OpenGL pipeline it's time to look to the
> future.
> This is where my problem lies; although there are many examples on the 'net
> showing me how to animate the most beautiful normal mapped cobbled cube, or
> tutorials on specific rendering / culling / lighting algorithms, I've yet to
> find a resource beside Michael Abrash's Graphics Programming Black Book
> (which sits on my desk, reminding me of what I should have been doing all
> along) which goes into the subject of writing a graphics engine. See
> although I can write perfectly passable C++, and thanks to my AS3
> programming experience which has frequently required that I get my projects
> right 'the first time', I'm fairly adept at planning and designing my
> classes before I begin... But from a C++ standpoint, I'm somewhat in the
> dark. What are the best practices for building a graphics engine in C++?
> Exactly how OOP should I go? How should I handle the use of multiple shaders
> on chunks of geometry? There's just this giant multitude of questions, and
> although I have a knack for over engineering things, I really want to have a
> better picture of what I'm aiming for before I start, or I'll end up with
> the monstrosity that is my terrain prototype (it runs, but it's so, so
> ugly).
> Currently my idea for a graphics engine involves something like an engine
> class that takes render packets of data, has some sort of ability to sort
> them into an order that best facilitates speedy rendering, and changes its
> state depending on the information in the packets. E.g. one packet might be
> a chunk of terrain, and could include information about what shaders to use,
> how to determine its potential visible set, and the type of collision
> detection to use (though this deviates a little from the graphics engine
> part). There might be an arbitrary number of terrain packets, followed by
> some character models or buildings using a different type of culling
> algorithm. My worry aside from the obvious problem of the whole idea
> possibly being crap, is that it's too general. How could I possibly create
> an engine that handles every possible rendering type, shouldn't I perhaps
> start with the assertion that I'm building a terrain engine, and base
> everything around rendering sphere like chunks of geometry with varying
> levels of detail? But on the same token, I don't just want to end up with
> another terrain engine prototype as seen on youtube(tm).
> So I suppose what I'm looking for are book suggestions, or perhaps
> open-sourced graphics engines that I could look at and learn from, or maybe
> I should just try and get a position as a junior-something-programmer and
> learn by doing (the trouble is there aren't any such positions available
> where I live, and relocation would be difficult to justify based upon the
> possibly low salary expectations of a junior position, I'd like at least
> some leverage). Ideas? Suggestions? Is this even the right place to ask such
> broad questions? I'm all ears.
> - Tommy
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com<mailto:Sweng-***@lists.midnightryder.com>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Tommy Brett]

Thanks for the replies guys, I should clarify that yes a graphics programmer
is what I would like to become, and if I turn out to be Pretty Good at it,
then authoring my own game engine would be an exceptional challenge to
overcome. How I'm going to get to that end goal, or even if the end goal is
feasible is what I'm trying to ascertain. I understand your point Nicholas;
the graphics engine I'd like to create might not even be rightfully called a
'graphics engine', but I need it to be something that would prove to an
employer that I'm capable of doing more than stitching together some
rendering algorithms I pulled out of a couple of GDC papers, and in doing
so, would get me into a position where I'm able to learn from the pros.

I've noticed that a lot of computer games degrees (of which I'm a graduate,
albeit 5 years ago) include courses specifically designed to build you a
portfolio, but almost all of them will involve a 3rd party tool to create
the work. For example, "get 5 people together and create a 1 level prototype
using Ogre, Unity or Torque". When I did my 'portfolio course' we actually
built our own engine from scratch (of which I built the physics, AI and
'game' part of) and our Professor echoed the same sentiments as those in
this email thread. But in that case we actually overcame the odds and
produced a pretty good piece of work that I was able to extend through
several other courses to include networking and more advanced modular AI...
But again, the code was not something I'd be overly proud of now, when
attempting to apply for a position that doesn't have the word 'junior' in
it.

>From this email discussion I'm hoping to be able to get a clear picture of
what I can do in the time that I have to create a piece of work I can show
to an employer and say "I built this, the code is solid although simple in
places, there's room for expansion but this proves without a doubt that I am
not only capable of working with modern graphics engines, but I also know
enough of the underlying low level concepts to have designed this". Which I
think would give me an advantage over other graduates, and give me leverage
for a paycheck that would justify leaving my current position.

- Tommy

On Fri, Mar 18, 2011 at 10:15 AM, Nicholas Aelick <***@gmail.com>wrote:

> I would tend to agree with Megan. Building a graphics engine from scratch
> for the learning experience is a little like building a computer by
> soldering bits of wire together for the learning experience. You're unlikely
> to get far, and even if you succeed, you're unlikely to end up with
> something nicer then is already available.
>
> I would recommend looking into existing solutions (partially because any
> place you work for will have you working with those, anyways). If you're
> really serious about building a custom graphics engine, find one that's open
> source, and see how they solved the problems you've been having.
>
> Nicholas
>
>
> On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <***@gmail.com> wrote:
>
>> To take it in another way, I would generally recommend no one set out to
>> write their own graphics engine from scratch without first using another.
>> Simple example - OGRE. Great jumping-off point, great community, etc.
>> Build up a game around it, then burrow down and start changing OGRE to suit
>> your needs, then burrow down further and tweak the graphics, and so on.
>>
>> There's very little reason to write a graphics engine yourself, unless
>> you're addressing the faults of a pre-existing engine you tried and found
>> insufficient. Without that grounding, you're stumbling in the dark, and it
>> will likely take quite some time and a hundred dead ends for you to get
>> within spitting distance of even the bad current graphics engines. You're
>> also needlessly re-inventing the wheel, unless Graphics Programming is
>> specifically what you're aiming at - there's a lot more to games programming
>> than rendering, and your portfolio would need to be more than rendered
>> planets to break in (ie. what happens on or with those planets, and why
>> they're fun to walk around / fly around / whatever).
>>
>> ----
>> Megan Fox
>> http://www.shalinor.com/
>> http://www.glassbottomgames.com/
>>
>>
>>
>> On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com>wrote:
>>
>>> Hi Tommy,
>>>
>>> While this might not answer completely such a broad question, I would
>>> recommend you look at this site:
>>> http://gamearchitect.net/
>>>
>>> More specifically, some articles called 'An Anatomy of Despair'
>>> explain the design of the author's engine. See for instance:
>>>
>>> http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views
>>>
>>> I would also recommend the excellent book 'Game Engine Architecture',
>>> by Jason Gregory.
>>>
>>> Note, however, that these resources are mainly oriented towards game
>>> engines as whole. AFAIK, there are not many resources on the specifics
>>> of graphics engine design and architecture but, as you said, mainly a
>>> lot of very technical articles one some technologies and
>>> implementations.
>>>
>>> For what it's worth, here is the structure I currently use on my
>>> personal engine. It works, but has not been confronted to a real prod,
>>> so what follows is definitely not bullet-proof.
>>>
>>> The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
>>> I use this layer mainly for readibility purposes (e.g being able to
>>> write texture->enableFiltering(BILINEAR) instead of two or three
>>> cryptic OGL/DX calls), but with a bit more work, you can also use it
>>> to make the engine cross-api (with many simple but error-prone
>>> details, such as matrix transpositions).
>>>
>>> On top of that, two modules are present:
>>> - The graphics scene contains an abstract description of graphical
>>> elements that are part of the world. No shading algorithms here, just
>>> plain buffers augmented with various information (transforms...). The
>>> scene handles all issues about traversal and culling. Whenever a
>>> render is asked, a render list is built by querying the spatial
>>> partitioning structure, and then sent to the renderer (see below).
>>> - The renderer is just the raw implementation of shading algorithms.
>>> It receives a render list + a frustum and draws everything, using
>>> various techniques (e.g forward or deferred shading).
>>>
>>> The render list basically contains packets in the form <transform,
>>> mesh, material> (+ lights), but this is highly dependent on how the
>>> rest is implemented.
>>>
>>> Hope this helps! I'll also be very happy to learn about other ways of
>>> designing graphics engines, if someone else has good resources on the
>>> subject ;)
>>>
>>> Rémi
>>>
>>> 2011/3/18 Tommy Brett <***@gmail.com>:
>>> > Hi folks,
>>> > I've been working as an AS3 programmer / animator for the last 3.5
>>> years,
>>> > and have the strongest desire to leave the marketing industry behind
>>> and
>>> > pursue my original intended career as a games programmer, which I
>>> sidelined
>>> > due to a very coincidental series of events. To this end, I've been
>>> (slowly)
>>> > building a graphics engine using C++, OpenGL and nVidia CG which
>>> renders
>>> > very large chunks of terrain using GPU Clipmaps. The idea is that
>>> eventually
>>> > it will render planets with atmosphere and real time shadows, or die
>>> trying,
>>> > and in any case will serve as my main portfolio piece when I eventually
>>> > start applying for positions. Because I've been outside of my
>>> programming
>>> > comfort zone for so long it's been slow going, but now that I've
>>> finally got
>>> > a firm grasp on shaders and the OpenGL pipeline it's time to look to
>>> the
>>> > future.
>>> > This is where my problem lies; although there are many examples on the
>>> 'net
>>> > showing me how to animate the most beautiful normal mapped cobbled
>>> cube, or
>>> > tutorials on specific rendering / culling / lighting algorithms, I've
>>> yet to
>>> > find a resource beside Michael Abrash's Graphics Programming Black Book
>>> > (which sits on my desk, reminding me of what I should have been doing
>>> all
>>> > along) which goes into the subject of writing a graphics engine. See
>>> > although I can write perfectly passable C++, and thanks to my AS3
>>> > programming experience which has frequently required that I get my
>>> projects
>>> > right 'the first time', I'm fairly adept at planning and designing my
>>> > classes before I begin... But from a C++ standpoint, I'm somewhat in
>>> the
>>> > dark. What are the best practices for building a graphics engine in
>>> C++?
>>> > Exactly how OOP should I go? How should I handle the use of multiple
>>> shaders
>>> > on chunks of geometry? There's just this giant multitude of questions,
>>> and
>>> > although I have a knack for over engineering things, I really want to
>>> have a
>>> > better picture of what I'm aiming for before I start, or I'll end up
>>> with
>>> > the monstrosity that is my terrain prototype (it runs, but it's so, so
>>> > ugly).
>>> > Currently my idea for a graphics engine involves something like an
>>> engine
>>> > class that takes render packets of data, has some sort of ability to
>>> sort
>>> > them into an order that best facilitates speedy rendering, and changes
>>> its
>>> > state depending on the information in the packets. E.g. one packet
>>> might be
>>> > a chunk of terrain, and could include information about what shaders to
>>> use,
>>> > how to determine its potential visible set, and the type of collision
>>> > detection to use (though this deviates a little from the graphics
>>> engine
>>> > part). There might be an arbitrary number of terrain packets, followed
>>> by
>>> > some character models or buildings using a different type of culling
>>> > algorithm. My worry aside from the obvious problem of the whole idea
>>> > possibly being crap, is that it's too general. How could I possibly
>>> create
>>> > an engine that handles every possible rendering type, shouldn't I
>>> perhaps
>>> > start with the assertion that I'm building a terrain engine, and base
>>> > everything around rendering sphere like chunks of geometry with varying
>>> > levels of detail? But on the same token, I don't just want to end up
>>> with
>>> > another terrain engine prototype as seen on youtube(tm).
>>> > So I suppose what I'm looking for are book suggestions, or perhaps
>>> > open-sourced graphics engines that I could look at and learn from, or
>>> maybe
>>> > I should just try and get a position as a junior-something-programmer
>>> and
>>> > learn by doing (the trouble is there aren't any such positions
>>> available
>>> > where I live, and relocation would be difficult to justify based upon
>>> the
>>> > possibly low salary expectations of a junior position, I'd like at
>>> least
>>> > some leverage). Ideas? Suggestions? Is this even the right place to ask
>>> such
>>> > broad questions? I'm all ears.
>>> > - Tommy
>>> > _______________________________________________
>>> > Sweng-Gamedev mailing list
>>> > Sweng-***@lists.midnightryder.com
>>> >
>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>> >
>>> >
>>> _______________________________________________
>>> Sweng-Gamedev mailing list
>>> Sweng-***@lists.midnightryder.com
>>>
>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>
>>
>> _______________________________________________
>> Sweng-Gamedev mailing list
>> Sweng-***@lists.midnightryder.com
>>
>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>
>>
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Tommy Brett]

There were a few emails between me beginning to write my reply and finishing
it.

Peter - I actually recall a thread on this mailing list last year which
described a programming paradigm that shifted away from OOP toward a more
data driven approach and it piqued my interest considerably. I believe it
separated data and code in an attempt to allow transformation functions to
work on streams of data, I can't find it in my inbox at the moment though.
What you said is basically what I aim to do - create a proof of concept that
a little more advanced than taking an existing 3rd party tool and creating
something with it. Thanks also to Oladotun for giving me some links to read.

- Tommy

On Fri, Mar 18, 2011 at 10:40 AM, Tommy Brett <***@gmail.com> wrote:

> Thanks for the replies guys, I should clarify that yes a graphics
> programmer is what I would like to become, and if I turn out to be Pretty
> Good at it, then authoring my own game engine would be an exceptional
> challenge to overcome. How I'm going to get to that end goal, or even if the
> end goal is feasible is what I'm trying to ascertain. I understand your
> point Nicholas; the graphics engine I'd like to create might not even be
> rightfully called a 'graphics engine', but I need it to be something that
> would prove to an employer that I'm capable of doing more than stitching
> together some rendering algorithms I pulled out of a couple of GDC papers,
> and in doing so, would get me into a position where I'm able to learn from
> the pros.
>
> I've noticed that a lot of computer games degrees (of which I'm a graduate,
> albeit 5 years ago) include courses specifically designed to build you a
> portfolio, but almost all of them will involve a 3rd party tool to create
> the work. For example, "get 5 people together and create a 1 level prototype
> using Ogre, Unity or Torque". When I did my 'portfolio course' we actually
> built our own engine from scratch (of which I built the physics, AI and
> 'game' part of) and our Professor echoed the same sentiments as those in
> this email thread. But in that case we actually overcame the odds and
> produced a pretty good piece of work that I was able to extend through
> several other courses to include networking and more advanced modular AI...
> But again, the code was not something I'd be overly proud of now, when
> attempting to apply for a position that doesn't have the word 'junior' in
> it.
>
> From this email discussion I'm hoping to be able to get a clear picture of
> what I can do in the time that I have to create a piece of work I can show
> to an employer and say "I built this, the code is solid although simple in
> places, there's room for expansion but this proves without a doubt that I am
> not only capable of working with modern graphics engines, but I also know
> enough of the underlying low level concepts to have designed this". Which I
> think would give me an advantage over other graduates, and give me leverage
> for a paycheck that would justify leaving my current position.
>
> - Tommy
>
> On Fri, Mar 18, 2011 at 10:15 AM, Nicholas Aelick <***@gmail.com>wrote:
>
>> I would tend to agree with Megan. Building a graphics engine from scratch
>> for the learning experience is a little like building a computer by
>> soldering bits of wire together for the learning experience. You're unlikely
>> to get far, and even if you succeed, you're unlikely to end up with
>> something nicer then is already available.
>>
>> I would recommend looking into existing solutions (partially because any
>> place you work for will have you working with those, anyways). If you're
>> really serious about building a custom graphics engine, find one that's open
>> source, and see how they solved the problems you've been having.
>>
>> Nicholas
>>
>>
>> On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <***@gmail.com> wrote:
>>
>>> To take it in another way, I would generally recommend no one set out to
>>> write their own graphics engine from scratch without first using another.
>>> Simple example - OGRE. Great jumping-off point, great community, etc.
>>> Build up a game around it, then burrow down and start changing OGRE to suit
>>> your needs, then burrow down further and tweak the graphics, and so on.
>>>
>>> There's very little reason to write a graphics engine yourself, unless
>>> you're addressing the faults of a pre-existing engine you tried and found
>>> insufficient. Without that grounding, you're stumbling in the dark, and it
>>> will likely take quite some time and a hundred dead ends for you to get
>>> within spitting distance of even the bad current graphics engines. You're
>>> also needlessly re-inventing the wheel, unless Graphics Programming is
>>> specifically what you're aiming at - there's a lot more to games programming
>>> than rendering, and your portfolio would need to be more than rendered
>>> planets to break in (ie. what happens on or with those planets, and why
>>> they're fun to walk around / fly around / whatever).
>>>
>>> ----
>>> Megan Fox
>>> http://www.shalinor.com/
>>> http://www.glassbottomgames.com/
>>>
>>>
>>>
>>> On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com>wrote:
>>>
>>>> Hi Tommy,
>>>>
>>>> While this might not answer completely such a broad question, I would
>>>> recommend you look at this site:
>>>> http://gamearchitect.net/
>>>>
>>>> More specifically, some articles called 'An Anatomy of Despair'
>>>> explain the design of the author's engine. See for instance:
>>>>
>>>> http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views
>>>>
>>>> I would also recommend the excellent book 'Game Engine Architecture',
>>>> by Jason Gregory.
>>>>
>>>> Note, however, that these resources are mainly oriented towards game
>>>> engines as whole. AFAIK, there are not many resources on the specifics
>>>> of graphics engine design and architecture but, as you said, mainly a
>>>> lot of very technical articles one some technologies and
>>>> implementations.
>>>>
>>>> For what it's worth, here is the structure I currently use on my
>>>> personal engine. It works, but has not been confronted to a real prod,
>>>> so what follows is definitely not bullet-proof.
>>>>
>>>> The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
>>>> I use this layer mainly for readibility purposes (e.g being able to
>>>> write texture->enableFiltering(BILINEAR) instead of two or three
>>>> cryptic OGL/DX calls), but with a bit more work, you can also use it
>>>> to make the engine cross-api (with many simple but error-prone
>>>> details, such as matrix transpositions).
>>>>
>>>> On top of that, two modules are present:
>>>> - The graphics scene contains an abstract description of graphical
>>>> elements that are part of the world. No shading algorithms here, just
>>>> plain buffers augmented with various information (transforms...). The
>>>> scene handles all issues about traversal and culling. Whenever a
>>>> render is asked, a render list is built by querying the spatial
>>>> partitioning structure, and then sent to the renderer (see below).
>>>> - The renderer is just the raw implementation of shading algorithms.
>>>> It receives a render list + a frustum and draws everything, using
>>>> various techniques (e.g forward or deferred shading).
>>>>
>>>> The render list basically contains packets in the form <transform,
>>>> mesh, material> (+ lights), but this is highly dependent on how the
>>>> rest is implemented.
>>>>
>>>> Hope this helps! I'll also be very happy to learn about other ways of
>>>> designing graphics engines, if someone else has good resources on the
>>>> subject ;)
>>>>
>>>> Rémi
>>>>
>>>> 2011/3/18 Tommy Brett <***@gmail.com>:
>>>> > Hi folks,
>>>> > I've been working as an AS3 programmer / animator for the last 3.5
>>>> years,
>>>> > and have the strongest desire to leave the marketing industry behind
>>>> and
>>>> > pursue my original intended career as a games programmer, which I
>>>> sidelined
>>>> > due to a very coincidental series of events. To this end, I've been
>>>> (slowly)
>>>> > building a graphics engine using C++, OpenGL and nVidia CG which
>>>> renders
>>>> > very large chunks of terrain using GPU Clipmaps. The idea is that
>>>> eventually
>>>> > it will render planets with atmosphere and real time shadows, or die
>>>> trying,
>>>> > and in any case will serve as my main portfolio piece when I
>>>> eventually
>>>> > start applying for positions. Because I've been outside of my
>>>> programming
>>>> > comfort zone for so long it's been slow going, but now that I've
>>>> finally got
>>>> > a firm grasp on shaders and the OpenGL pipeline it's time to look to
>>>> the
>>>> > future.
>>>> > This is where my problem lies; although there are many examples on the
>>>> 'net
>>>> > showing me how to animate the most beautiful normal mapped cobbled
>>>> cube, or
>>>> > tutorials on specific rendering / culling / lighting algorithms, I've
>>>> yet to
>>>> > find a resource beside Michael Abrash's Graphics Programming Black
>>>> Book
>>>> > (which sits on my desk, reminding me of what I should have been doing
>>>> all
>>>> > along) which goes into the subject of writing a graphics engine. See
>>>> > although I can write perfectly passable C++, and thanks to my AS3
>>>> > programming experience which has frequently required that I get my
>>>> projects
>>>> > right 'the first time', I'm fairly adept at planning and designing my
>>>> > classes before I begin... But from a C++ standpoint, I'm somewhat in
>>>> the
>>>> > dark. What are the best practices for building a graphics engine in
>>>> C++?
>>>> > Exactly how OOP should I go? How should I handle the use of multiple
>>>> shaders
>>>> > on chunks of geometry? There's just this giant multitude of questions,
>>>> and
>>>> > although I have a knack for over engineering things, I really want to
>>>> have a
>>>> > better picture of what I'm aiming for before I start, or I'll end up
>>>> with
>>>> > the monstrosity that is my terrain prototype (it runs, but it's so, so
>>>> > ugly).
>>>> > Currently my idea for a graphics engine involves something like an
>>>> engine
>>>> > class that takes render packets of data, has some sort of ability to
>>>> sort
>>>> > them into an order that best facilitates speedy rendering, and changes
>>>> its
>>>> > state depending on the information in the packets. E.g. one packet
>>>> might be
>>>> > a chunk of terrain, and could include information about what shaders
>>>> to use,
>>>> > how to determine its potential visible set, and the type of collision
>>>> > detection to use (though this deviates a little from the graphics
>>>> engine
>>>> > part). There might be an arbitrary number of terrain packets, followed
>>>> by
>>>> > some character models or buildings using a different type of culling
>>>> > algorithm. My worry aside from the obvious problem of the whole idea
>>>> > possibly being crap, is that it's too general. How could I possibly
>>>> create
>>>> > an engine that handles every possible rendering type, shouldn't I
>>>> perhaps
>>>> > start with the assertion that I'm building a terrain engine, and base
>>>> > everything around rendering sphere like chunks of geometry with
>>>> varying
>>>> > levels of detail? But on the same token, I don't just want to end up
>>>> with
>>>> > another terrain engine prototype as seen on youtube(tm).
>>>> > So I suppose what I'm looking for are book suggestions, or perhaps
>>>> > open-sourced graphics engines that I could look at and learn from, or
>>>> maybe
>>>> > I should just try and get a position as a junior-something-programmer
>>>> and
>>>> > learn by doing (the trouble is there aren't any such positions
>>>> available
>>>> > where I live, and relocation would be difficult to justify based upon
>>>> the
>>>> > possibly low salary expectations of a junior position, I'd like at
>>>> least
>>>> > some leverage). Ideas? Suggestions? Is this even the right place to
>>>> ask such
>>>> > broad questions? I'm all ears.
>>>> > - Tommy
>>>> > _______________________________________________
>>>> > Sweng-Gamedev mailing list
>>>> > Sweng-***@lists.midnightryder.com
>>>> >
>>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>> >
>>>> >
>>>> _______________________________________________
>>>> Sweng-Gamedev mailing list
>>>> Sweng-***@lists.midnightryder.com
>>>>
>>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>>
>>>
>>> _______________________________________________
>>> Sweng-Gamedev mailing list
>>> Sweng-***@lists.midnightryder.com
>>>
>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>
>>>
>>
>> _______________________________________________
>> Sweng-Gamedev mailing list
>> Sweng-***@lists.midnightryder.com
>>
>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>
>>
>

[Steven Kah Hien Wong]

You can learn by rolling your own or by observing/working with others. Both
is even better! :) So it might be a good time to explore the latter (Ogre is
a start).

Personally, as I worked with more engines (Unreal, Evolution, Quake, Ogre
and internal ones), certain patterns and similarities became apparent. This
knowledge helped immensely in my own designs. It's a long and iterative
process, but I believe it's necessary.

Being able to create useful, low-level modifications to a complicated,
existing engine also speaks volumes to potential employers. Reading,
deeply-understanding and working with other people's code is critical in
team projects. I would argue doing that is at least just as hard as writing
your own (including fighting the temptation that your own version would be
"better").

But I will admit, it's not as fun as rolling your own. :)

My 5c!

-Steven


On 18 March 2011 10:46, Tommy Brett <***@gmail.com> wrote:

> There were a few emails between me beginning to write my reply and
> finishing it.
>
> Peter - I actually recall a thread on this mailing list last year which
> described a programming paradigm that shifted away from OOP toward a more
> data driven approach and it piqued my interest considerably. I believe it
> separated data and code in an attempt to allow transformation functions to
> work on streams of data, I can't find it in my inbox at the moment though.
> What you said is basically what I aim to do - create a proof of concept that
> a little more advanced than taking an existing 3rd party tool and creating
> something with it. Thanks also to Oladotun for giving me some links to read.
>
> - Tommy
>
>
> On Fri, Mar 18, 2011 at 10:40 AM, Tommy Brett <***@gmail.com>wrote:
>
>> Thanks for the replies guys, I should clarify that yes a graphics
>> programmer is what I would like to become, and if I turn out to be Pretty
>> Good at it, then authoring my own game engine would be an exceptional
>> challenge to overcome. How I'm going to get to that end goal, or even if the
>> end goal is feasible is what I'm trying to ascertain. I understand your
>> point Nicholas; the graphics engine I'd like to create might not even be
>> rightfully called a 'graphics engine', but I need it to be something that
>> would prove to an employer that I'm capable of doing more than stitching
>> together some rendering algorithms I pulled out of a couple of GDC papers,
>> and in doing so, would get me into a position where I'm able to learn from
>> the pros.
>>
>> I've noticed that a lot of computer games degrees (of which I'm a
>> graduate, albeit 5 years ago) include courses specifically designed to build
>> you a portfolio, but almost all of them will involve a 3rd party tool to
>> create the work. For example, "get 5 people together and create a 1 level
>> prototype using Ogre, Unity or Torque". When I did my 'portfolio course' we
>> actually built our own engine from scratch (of which I built the physics, AI
>> and 'game' part of) and our Professor echoed the same sentiments as those in
>> this email thread. But in that case we actually overcame the odds and
>> produced a pretty good piece of work that I was able to extend through
>> several other courses to include networking and more advanced modular AI...
>> But again, the code was not something I'd be overly proud of now, when
>> attempting to apply for a position that doesn't have the word 'junior' in
>> it.
>>
>> From this email discussion I'm hoping to be able to get a clear picture of
>> what I can do in the time that I have to create a piece of work I can show
>> to an employer and say "I built this, the code is solid although simple in
>> places, there's room for expansion but this proves without a doubt that I am
>> not only capable of working with modern graphics engines, but I also know
>> enough of the underlying low level concepts to have designed this". Which I
>> think would give me an advantage over other graduates, and give me leverage
>> for a paycheck that would justify leaving my current position.
>>
>> - Tommy
>>
>> On Fri, Mar 18, 2011 at 10:15 AM, Nicholas Aelick <***@gmail.com>wrote:
>>
>>> I would tend to agree with Megan. Building a graphics engine from scratch
>>> for the learning experience is a little like building a computer by
>>> soldering bits of wire together for the learning experience. You're unlikely
>>> to get far, and even if you succeed, you're unlikely to end up with
>>> something nicer then is already available.
>>>
>>> I would recommend looking into existing solutions (partially because any
>>> place you work for will have you working with those, anyways). If you're
>>> really serious about building a custom graphics engine, find one that's open
>>> source, and see how they solved the problems you've been having.
>>>
>>> Nicholas
>>>
>>>
>>> On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <***@gmail.com> wrote:
>>>
>>>> To take it in another way, I would generally recommend no one set out to
>>>> write their own graphics engine from scratch without first using another.
>>>> Simple example - OGRE. Great jumping-off point, great community, etc.
>>>> Build up a game around it, then burrow down and start changing OGRE to suit
>>>> your needs, then burrow down further and tweak the graphics, and so on.
>>>>
>>>> There's very little reason to write a graphics engine yourself, unless
>>>> you're addressing the faults of a pre-existing engine you tried and found
>>>> insufficient. Without that grounding, you're stumbling in the dark, and it
>>>> will likely take quite some time and a hundred dead ends for you to get
>>>> within spitting distance of even the bad current graphics engines. You're
>>>> also needlessly re-inventing the wheel, unless Graphics Programming is
>>>> specifically what you're aiming at - there's a lot more to games programming
>>>> than rendering, and your portfolio would need to be more than rendered
>>>> planets to break in (ie. what happens on or with those planets, and why
>>>> they're fun to walk around / fly around / whatever).
>>>>
>>>> ----
>>>> Megan Fox
>>>> http://www.shalinor.com/
>>>> http://www.glassbottomgames.com/
>>>>
>>>>
>>>>
>>>> On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <***@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi Tommy,
>>>>>
>>>>> While this might not answer completely such a broad question, I would
>>>>> recommend you look at this site:
>>>>> http://gamearchitect.net/
>>>>>
>>>>> More specifically, some articles called 'An Anatomy of Despair'
>>>>> explain the design of the author's engine. See for instance:
>>>>>
>>>>> http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views
>>>>>
>>>>> I would also recommend the excellent book 'Game Engine Architecture',
>>>>> by Jason Gregory.
>>>>>
>>>>> Note, however, that these resources are mainly oriented towards game
>>>>> engines as whole. AFAIK, there are not many resources on the specifics
>>>>> of graphics engine design and architecture but, as you said, mainly a
>>>>> lot of very technical articles one some technologies and
>>>>> implementations.
>>>>>
>>>>> For what it's worth, here is the structure I currently use on my
>>>>> personal engine. It works, but has not been confronted to a real prod,
>>>>> so what follows is definitely not bullet-proof.
>>>>>
>>>>> The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
>>>>> I use this layer mainly for readibility purposes (e.g being able to
>>>>> write texture->enableFiltering(BILINEAR) instead of two or three
>>>>> cryptic OGL/DX calls), but with a bit more work, you can also use it
>>>>> to make the engine cross-api (with many simple but error-prone
>>>>> details, such as matrix transpositions).
>>>>>
>>>>> On top of that, two modules are present:
>>>>> - The graphics scene contains an abstract description of graphical
>>>>> elements that are part of the world. No shading algorithms here, just
>>>>> plain buffers augmented with various information (transforms...). The
>>>>> scene handles all issues about traversal and culling. Whenever a
>>>>> render is asked, a render list is built by querying the spatial
>>>>> partitioning structure, and then sent to the renderer (see below).
>>>>> - The renderer is just the raw implementation of shading algorithms.
>>>>> It receives a render list + a frustum and draws everything, using
>>>>> various techniques (e.g forward or deferred shading).
>>>>>
>>>>> The render list basically contains packets in the form <transform,
>>>>> mesh, material> (+ lights), but this is highly dependent on how the
>>>>> rest is implemented.
>>>>>
>>>>> Hope this helps! I'll also be very happy to learn about other ways of
>>>>> designing graphics engines, if someone else has good resources on the
>>>>> subject ;)
>>>>>
>>>>> Rémi
>>>>>
>>>>> 2011/3/18 Tommy Brett <***@gmail.com>:
>>>>> > Hi folks,
>>>>> > I've been working as an AS3 programmer / animator for the last 3.5
>>>>> years,
>>>>> > and have the strongest desire to leave the marketing industry behind
>>>>> and
>>>>> > pursue my original intended career as a games programmer, which I
>>>>> sidelined
>>>>> > due to a very coincidental series of events. To this end, I've been
>>>>> (slowly)
>>>>> > building a graphics engine using C++, OpenGL and nVidia CG which
>>>>> renders
>>>>> > very large chunks of terrain using GPU Clipmaps. The idea is that
>>>>> eventually
>>>>> > it will render planets with atmosphere and real time shadows, or die
>>>>> trying,
>>>>> > and in any case will serve as my main portfolio piece when I
>>>>> eventually
>>>>> > start applying for positions. Because I've been outside of my
>>>>> programming
>>>>> > comfort zone for so long it's been slow going, but now that I've
>>>>> finally got
>>>>> > a firm grasp on shaders and the OpenGL pipeline it's time to look to
>>>>> the
>>>>> > future.
>>>>> > This is where my problem lies; although there are many examples on
>>>>> the 'net
>>>>> > showing me how to animate the most beautiful normal mapped cobbled
>>>>> cube, or
>>>>> > tutorials on specific rendering / culling / lighting algorithms, I've
>>>>> yet to
>>>>> > find a resource beside Michael Abrash's Graphics Programming Black
>>>>> Book
>>>>> > (which sits on my desk, reminding me of what I should have been doing
>>>>> all
>>>>> > along) which goes into the subject of writing a graphics engine. See
>>>>> > although I can write perfectly passable C++, and thanks to my AS3
>>>>> > programming experience which has frequently required that I get my
>>>>> projects
>>>>> > right 'the first time', I'm fairly adept at planning and designing my
>>>>> > classes before I begin... But from a C++ standpoint, I'm somewhat in
>>>>> the
>>>>> > dark. What are the best practices for building a graphics engine in
>>>>> C++?
>>>>> > Exactly how OOP should I go? How should I handle the use of multiple
>>>>> shaders
>>>>> > on chunks of geometry? There's just this giant multitude of
>>>>> questions, and
>>>>> > although I have a knack for over engineering things, I really want to
>>>>> have a
>>>>> > better picture of what I'm aiming for before I start, or I'll end up
>>>>> with
>>>>> > the monstrosity that is my terrain prototype (it runs, but it's so,
>>>>> so
>>>>> > ugly).
>>>>> > Currently my idea for a graphics engine involves something like an
>>>>> engine
>>>>> > class that takes render packets of data, has some sort of ability to
>>>>> sort
>>>>> > them into an order that best facilitates speedy rendering, and
>>>>> changes its
>>>>> > state depending on the information in the packets. E.g. one packet
>>>>> might be
>>>>> > a chunk of terrain, and could include information about what shaders
>>>>> to use,
>>>>> > how to determine its potential visible set, and the type of collision
>>>>> > detection to use (though this deviates a little from the graphics
>>>>> engine
>>>>> > part). There might be an arbitrary number of terrain packets,
>>>>> followed by
>>>>> > some character models or buildings using a different type of culling
>>>>> > algorithm. My worry aside from the obvious problem of the whole idea
>>>>> > possibly being crap, is that it's too general. How could I possibly
>>>>> create
>>>>> > an engine that handles every possible rendering type, shouldn't I
>>>>> perhaps
>>>>> > start with the assertion that I'm building a terrain engine, and base
>>>>> > everything around rendering sphere like chunks of geometry with
>>>>> varying
>>>>> > levels of detail? But on the same token, I don't just want to end up
>>>>> with
>>>>> > another terrain engine prototype as seen on youtube(tm).
>>>>> > So I suppose what I'm looking for are book suggestions, or perhaps
>>>>> > open-sourced graphics engines that I could look at and learn from, or
>>>>> maybe
>>>>> > I should just try and get a position as a junior-something-programmer
>>>>> and
>>>>> > learn by doing (the trouble is there aren't any such positions
>>>>> available
>>>>> > where I live, and relocation would be difficult to justify based upon
>>>>> the
>>>>> > possibly low salary expectations of a junior position, I'd like at
>>>>> least
>>>>> > some leverage). Ideas? Suggestions? Is this even the right place to
>>>>> ask such
>>>>> > broad questions? I'm all ears.
>>>>> > - Tommy
>>>>> > _______________________________________________
>>>>> > Sweng-Gamedev mailing list
>>>>> > Sweng-***@lists.midnightryder.com
>>>>> >
>>>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>>> >
>>>>> >
>>>>> _______________________________________________
>>>>> Sweng-Gamedev mailing list
>>>>> Sweng-***@lists.midnightryder.com
>>>>>
>>>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>>>
>>>>
>>>> _______________________________________________
>>>> Sweng-Gamedev mailing list
>>>> Sweng-***@lists.midnightryder.com
>>>>
>>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Sweng-Gamedev mailing list
>>> Sweng-***@lists.midnightryder.com
>>>
>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>
>>>
>>
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Richard]

In article <AANLkTimvsC-***@mail.gmail.com>,
Steven Kah Hien Wong <***@epicfinalboss.com> writes:

> [...] I would argue doing that is at least just as hard as writing
> your own (including fighting the temptation that your own version would be
> "better").

Its exactly because of that temptation that working with other
people's code is harder than working with your own code. Working
productively with other people's code without rewriting it ("because I
can do better") demonstrates one necessary skill for working on a
team. You will need other skills, of course, but if you can't work
productively with other people's code, you'll never work out on a
team.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
<http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>

Legalize Adulthood! <http://legalizeadulthood.wordpress.com>

[Massimo Del Zotto]

2011/3/18 Tommy Brett <***@gmail.com>
>
> ... pursue my original intended career as a games programmer, ... it will
> render planets with atmosphere and real time shadows, or die trying, and in
> any case will serve as my main portfolio piece when I eventually start
> applying for positions.
>
Hello Tommy, I also started working on my engine for this same reason ("as
portfolio"). I don't know how other people are set but I suppose you will do
more or less what I've been doing up to now so I hope you will find my
advices useful.


> What are the best practices for building a graphics engine in C++? Exactly
> how OOP should I go?
>
A stupid answer would be "as far as you need but no more". When I started
working on my system I tried to make pretty much of everything OOP. I
eventually figured out that OOP is not always the best tool for the job.
Some systems are natively data-oriented and some will inevitably need
reinterpret_cast (oh, the horror). You will have to get your hands dirty. In
some cases, having less abstraction can be helpful. In general, my design
rarely relies on objects but is really based on "subsystems" which (as an
implementative detail) emerge from aggregating objects. Rather than objects,
I really think in terms of "protocols" (quite similar to "network
protocols"). You might say this is exactly what objects are made for. Yes
but the whole point is: don't be afraid of steering away from OOP every once
in a while.
The more you go "high level" the more OOP will result useful but at low
level, bad surprises might await you and the systems have to be coupled
somehow anyway.

As for how to write good C++ engines I'd suggest to remember that while C++
has no explicit "interface" classes, they are indeed there and they will
help you a lot (the compiler also seems to like them pretty much IMHO). Many
people seem to forget this so just in case...



> How should I handle the use of multiple shaders on chunks of geometry?
>
You're not going to solve this right now. Use iterative design. You really
have no choice, this would cause you to meltdown your head. I don't quite
figure out what you mean by "multiple shaders on chunks of geometry". Are
you referring to multiple materials? Are you referring to multipass
techniques?
Your first step is surely to figure out what you mean by "shader" because
when you have an engine, "shaders" as most APIs provide will probably lack
something here or there. For example my "engine shaders" provide special
binding points to link engine logic. Say you want to make everything fade to
black progressively as player's health drops to 0.
They also include additional information which are not part of the "API
shader" such as blend mode to use (this is similar to DirectX "effects") and
other additional stuff.
Now, those requirements weren't introduced in a day but evolved out of the
needs. This is why some people advises to "write games not engines"... you
really need to understand what you need for real before you can hope to
understand a way to solve the problem in a reusable, "engine" way.



> Currently my idea for a graphics engine involves something like an engine
> class that takes render packets of data, has some sort of ability to sort
> them into an order that best facilitates speedy rendering, and changes its
> state depending on the information in the packets.
>
This makes me think about
http://realtimecollisiondetection.net/blog/?p=86
but you probably don't mean this kind of low-level sort... or do you?
According to what you write next, it seems you're thinking at a much higher
level which apparently seems to be scenegraph's responsability. I don't
know.
As a final note here, the link I provided is effective but until you don't
finalize the systems involved it creates a strong dependancy which will slow
you down (where in the hierarchy should I put this batch?). Start easy
first! Leave the order emerge from your design first. Benchmark. Refine.
Repeat. Be aware of premature optimization. When in doubt write easy to
understand code, the bugs will generally emerge some time later, often
because of change in requirements, and you want the code to NOT bite you.
You probably have plenty of perf anyway and your first objective is get the
work done.



> So I suppose what I'm looking for are book suggestions, or perhaps
> open-sourced graphics engines that I could look at and learn from ... the
> trouble is there aren't any such positions available where I live, and
> relocation would be difficult to justify based upon the possibly low salary
> expectations of a junior position
>
It is always a good idea to consider already available engines. You really
have to look around you (at least your state as very minimum) and figure
out what to do as there's a serious social aspect to consider here.
From the few demos I've given out I can say that presenting something which
is 100% yours is fairly impressive, especially for small realities, it seems
that the idea of having somebody which can work out problems top-to-bottom
is very appreciated. Problem is that it takes a lot - a lot - of effort to
get to the first demo. It still takes a lot to go to the second as you will
keep working on it which in fact will result in less demos (or reuse of the
very same one).
Second problem is, as other people have said, turning a pre-made engine into
something yours is indeed valuable skill as well, which is likely much more
important if you're going to apply in consolidated companies. Not my case
here... :-(

Pre-made engines give you a great kickstart (it's awesome for morale)! On
the cons, I have to admit some projects I've looked at seemed to be halting
after a while. Let's not hide the fact: once you choose an engine you choose
a line of thinking and this implies you will think your design in a certain
way. Which is easier. The day you get a problem which is not directly
solvable by the engine, you start having issues.
I say this because a friend of mine had exactly this problem. Its
initial enthusiasm of having "parallax mapping and shadows everywhere"
quickly faded when he figured out the hard truth: the engine prevented him
from understanding the problems involved.
Of course, if you take your time to make sure you know what you're doing,
this should not be a problem and you'll get best of both worlds. I just
wanted to drop this warning.

Experimenting a bit with modding games such as Unreal or Quake is very
useful IMHO. If you haven't tried making a Quake map then you should really
try right now. Take a look at the formats. Do some black box analysis. Take
your time... if you can afford it.
Also try to figure out what assets you have access to and make sure you
understand the data encoded in ("know your data"). I've scratched my head
more than a year in making a shader work according to a "reference
implementation" before figuring out that the "reference" asset pipeline is
way smarter than I originally believed, let's say it works around data
format's dumbness...


I hope this helps,
Massimo

[Tommy Brett]

On Fri, Mar 18, 2011 at 1:30 PM, Massimo Del Zotto <***@gmail.com>wrote:

> 2011/3/18 Tommy Brett <***@gmail.com>
>
>> ... pursue my original intended career as a games programmer, ... it will
>> render planets with atmosphere and real time shadows, or die trying, and in
>> any case will serve as my main portfolio piece when I eventually start
>> applying for positions.
>>
> Hello Tommy, I also started working on my engine for this same reason ("as
> portfolio"). I don't know how other people are set but I suppose you will do
> more or less what I've been doing up to now so I hope you will find my
> advices useful.
>
>
>> What are the best practices for building a graphics engine in C++? Exactly
>> how OOP should I go?
>>
> A stupid answer would be "as far as you need but no more". When I started
> working on my system I tried to make pretty much of everything OOP. I
> eventually figured out that OOP is not always the best tool for the job.
> Some systems are natively data-oriented and some will inevitably need
> reinterpret_cast (oh, the horror). You will have to get your hands dirty. In
> some cases, having less abstraction can be helpful. In general, my design
> rarely relies on objects but is really based on "subsystems" which (as an
> implementative detail) emerge from aggregating objects. Rather than objects,
> I really think in terms of "protocols" (quite similar to "network
> protocols"). You might say this is exactly what objects are made for. Yes
> but the whole point is: don't be afraid of steering away from OOP every once
> in a while.
> The more you go "high level" the more OOP will result useful but at low
> level, bad surprises might await you and the systems have to be coupled
> somehow anyway.
>

This is why I mentioned 'how much is too much', because from my own
experimentation and from reading previous gamedev mailing list discussions,
OOP is clearly not always the best method for graphics engines.


> As for how to write good C++ engines I'd suggest to remember that while C++
> has no explicit "interface" classes, they are indeed there and they will
> help you a lot (the compiler also seems to like them pretty much IMHO). Many
> people seem to forget this so just in case...
>

Interfaces are definitely something I've gotten comfortable with using in my
AS3 programming, it's a neat alternative to deriving from a base class.


>
>
>
>> How should I handle the use of multiple shaders on chunks of geometry?
>>
> You're not going to solve this right now. Use iterative design. You really
> have no choice, this would cause you to meltdown your head. I don't quite
> figure out what you mean by "multiple shaders on chunks of geometry". Are
> you referring to multiple materials? Are you referring to multipass
> techniques?
> Your first step is surely to figure out what you mean by "shader" because
> when you have an engine, "shaders" as most APIs provide will probably lack
> something here or there. For example my "engine shaders" provide special
> binding points to link engine logic. Say you want to make everything fade to
> black progressively as player's health drops to 0.
> They also include additional information which are not part of the "API
> shader" such as blend mode to use (this is similar to DirectX "effects") and
> other additional stuff.
> Now, those requirements weren't introduced in a day but evolved out of the
> needs. This is why some people advises to "write games not engines"... you
> really need to understand what you need for real before you can hope to
> understand a way to solve the problem in a reusable, "engine" way.
>

Both materials and multi pass rendering techniques. I suppose it's a broad
question and I think you're right with the 'just code it' approach. What I
meant was, given a collection of meshes, each with their own textures and
effects, what would be the best way to organize those chunks of geometry
data, how would I manage engine state from chunk to chunk? E.g. switching
shaders, would switching shaders mid render even be viable?


>
>
>
>> Currently my idea for a graphics engine involves something like an engine
>> class that takes render packets of data, has some sort of ability to sort
>> them into an order that best facilitates speedy rendering, and changes its
>> state depending on the information in the packets.
>>
> This makes me think about
> http://realtimecollisiondetection.net/blog/?p=86
> but you probably don't mean this kind of low-level sort... or do you?
> According to what you write next, it seems you're thinking at a much higher
> level which apparently seems to be scenegraph's responsability. I don't
> know.
> As a final note here, the link I provided is effective but until you don't
> finalize the systems involved it creates a strong dependancy which will slow
> you down (where in the hierarchy should I put this batch?). Start easy
> first! Leave the order emerge from your design first. Benchmark. Refine.
> Repeat. Be aware of premature optimization. When in doubt write easy to
> understand code, the bugs will generally emerge some time later, often
> because of change in requirements, and you want the code to NOT bite you.
> You probably have plenty of perf anyway and your first objective is get the
> work done.
>
>
>
>> So I suppose what I'm looking for are book suggestions, or perhaps
>> open-sourced graphics engines that I could look at and learn from ... the
>> trouble is there aren't any such positions available where I live, and
>> relocation would be difficult to justify based upon the possibly low salary
>> expectations of a junior position
>>
> It is always a good idea to consider already available engines. You really
> have to look around you (at least your state as very minimum) and figure
> out what to do as there's a serious social aspect to consider here.
> From the few demos I've given out I can say that presenting something which
> is 100% yours is fairly impressive, especially for small realities, it seems
> that the idea of having somebody which can work out problems top-to-bottom
> is very appreciated. Problem is that it takes a lot - a lot - of effort to
> get to the first demo. It still takes a lot to go to the second as you will
> keep working on it which in fact will result in less demos (or reuse of the
> very same one).
> Second problem is, as other people have said, turning a pre-made engine
> into something yours is indeed valuable skill as well, which is likely much
> more important if you're going to apply in consolidated companies. Not my
> case here... :-(
>
> Pre-made engines give you a great kickstart (it's awesome for morale)! On
> the cons, I have to admit some projects I've looked at seemed to be halting
> after a while. Let's not hide the fact: once you choose an engine you choose
> a line of thinking and this implies you will think your design in a certain
> way. Which is easier. The day you get a problem which is not directly
> solvable by the engine, you start having issues.
> I say this because a friend of mine had exactly this problem. Its
> initial enthusiasm of having "parallax mapping and shadows everywhere"
> quickly faded when he figured out the hard truth: the engine prevented him
> from understanding the problems involved.
> Of course, if you take your time to make sure you know what you're doing,
> this should not be a problem and you'll get best of both worlds. I just
> wanted to drop this warning.
>

Your reply and other people's has given me a perspective that I didn't
consider before - taking someone elses engine and modifying it to suit my
own needs would certainly be an exemplary show of skill, especially for
studios that use 3rd party tools themselves


>
> Experimenting a bit with modding games such as Unreal or Quake is very
> useful IMHO. If you haven't tried making a Quake map then you should really
> try right now. Take a look at the formats. Do some black box analysis. Take
> your time... if you can afford it.
> Also try to figure out what assets you have access to and make sure you
> understand the data encoded in ("know your data"). I've scratched my head
> more than a year in making a shader work according to a "reference
> implementation" before figuring out that the "reference" asset pipeline is
> way smarter than I originally believed, let's say it works around data
> format's dumbness...
>

Actually it's funny you mention Quake mapping, when I started fiddling with
games and programming, I started with programming q-basic and making Doom
maps, which eventually progressed to Quake and Half-Life (I rather enjoyed
Worldcraft, even though it took forever and a day for them to release decent
texture mapping tools) and ignited my desire to become a programmer (luckily
I was young enough at the time that abusing goto didn't stick with me for
long). It's also why I have Michael Abrash's Black Book, I'm fascinated by
fps engines and how they calculate potentially visible data from a vast map.


>
>
> I hope this helps,
> Massimo
>

Greatly so, thank you.


>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Nathan Martz]

So much good advice on this thread. I'm loathe to add to it, but I want
to highlight a "big picture" trend that might help give you context and
inform how you proceed, especially if you decide to focus on rendering
architecture and write your own from the ground up. Here it is...



Over time rendering will be less and less about the care and feeding of
GPUs and more about (embarrassingly) parallel computation. For the last
decade and change, the dominant model in graphics has been "CPU makes a
big list of requests, GPU turns them into pixels." That's still sort of
true, but becoming less so every day. As CPUs gain cores and as GPUs
become increasingly programmable and CPU like, rendering is becoming all
about deciding how you divide up work across a series of (usually
heterogeneous) processors. At the start of this console generation,
people started spinning the renderer off onto a separate thread from the
main game. A few years later, console and PC engines started to use
multiple threads just for rendering (usually some sort of parallel
command buffer generation). In the past few years (earlier on the PS3),
people started adopting hybrid CPU/GPU techniques, moving "GPU work"
like post-processing, skinning, anti-aliasing, lighting, occlusion, etc.
off the GPU and onto the CPU (especially SPUs). When you look at the
really cutting edge stuff, especially in the world of DX10/11, you see
even more of complex CPU/GPU integration. Over time, GPUs and CPUs will
be even harder to tell apart and provide more and more options for how
do divide the work between them. It's an awesome, but also scarily
complex and open ended future.



The one thing I can say with confidence is that to be competitive,
future renderers will need to do the majority of their computation in
parallel across multiple processors. And that's where the really
important, but also really not easy problems are. As we all know,
parallel programming is really tricky, doubly so if you are starting
with a big renderer that was never designed with it in mind.
Fortunately, there's tons of great literature on the topic (much of
which like Data Oriented Design and Mike Acton's publications have
already been mentioned). It's definitely something you should be
thinking and reading about if you want to try your hand at writing your
own renderer. Of course, there's a universe of places to start and
things to explore, but if your goal is to land a job at a high end game
dev studio, there are, for my money, few more impressive things you
could demonstrate than a renderer that is thoughtfully and productively
parallel.



Good luck with whatever you decide to do!

-Nathan



From: sweng-gamedev-***@lists.midnightryder.com
[mailto:sweng-gamedev-***@lists.midnightryder.com] On Behalf Of
Tommy Brett
Sent: Friday, March 18, 2011 11:26 AM
To: sweng-***@midnightryder.com
Subject: Re: [Sweng-Gamedev] Architecting and programming a graphics
engine in C++ & OpenGL





On Fri, Mar 18, 2011 at 1:30 PM, Massimo Del Zotto
<***@gmail.com> wrote:

2011/3/18 Tommy Brett <***@gmail.com>

... pursue my original intended career as a games programmer,
... it will render planets with atmosphere and real time shadows, or die
trying, and in any case will serve as my main portfolio piece when I
eventually start applying for positions.

Hello Tommy, I also started working on my engine for this same reason
("as portfolio"). I don't know how other people are set but I suppose
you will do more or less what I've been doing up to now so I hope you
will find my advices useful.



What are the best practices for building a graphics engine in
C++? Exactly how OOP should I go?

A stupid answer would be "as far as you need but no more". When I
started working on my system I tried to make pretty much of everything
OOP. I eventually figured out that OOP is not always the best tool for
the job. Some systems are natively data-oriented and some will
inevitably need reinterpret_cast (oh, the horror). You will have to get
your hands dirty. In some cases, having less abstraction can be helpful.
In general, my design rarely relies on objects but is really based on
"subsystems" which (as an implementative detail) emerge from aggregating
objects. Rather than objects, I really think in terms of "protocols"
(quite similar to "network protocols"). You might say this is exactly
what objects are made for. Yes but the whole point is: don't be afraid
of steering away from OOP every once in a while.

The more you go "high level" the more OOP will result useful but at low
level, bad surprises might await you and the systems have to be coupled
somehow anyway.



This is why I mentioned 'how much is too much', because from my own
experimentation and from reading previous gamedev mailing list
discussions, OOP is clearly not always the best method for graphics
engines.



As for how to write good C++ engines I'd suggest to remember
that while C++ has no explicit "interface" classes, they are indeed
there and they will help you a lot (the compiler also seems to like them
pretty much IMHO). Many people seem to forget this so just in case...



Interfaces are definitely something I've gotten comfortable with using
in my AS3 programming, it's a neat alternative to deriving from a base
class.







How should I handle the use of multiple shaders on
chunks of geometry?

You're not going to solve this right now. Use iterative design.
You really have no choice, this would cause you to meltdown your head. I
don't quite figure out what you mean by "multiple shaders on chunks of
geometry". Are you referring to multiple materials? Are you referring to
multipass techniques?

Your first step is surely to figure out what you mean by
"shader" because when you have an engine, "shaders" as most APIs provide
will probably lack something here or there. For example my "engine
shaders" provide special binding points to link engine logic. Say you
want to make everything fade to black progressively as player's health
drops to 0.

They also include additional information which are not part of
the "API shader" such as blend mode to use (this is similar to DirectX
"effects") and other additional stuff.

Now, those requirements weren't introduced in a day but evolved
out of the needs. This is why some people advises to "write games not
engines"... you really need to understand what you need for real before
you can hope to understand a way to solve the problem in a reusable,
"engine" way.



Both materials and multi pass rendering techniques. I suppose it's a
broad question and I think you're right with the 'just code it'
approach. What I meant was, given a collection of meshes, each with
their own textures and effects, what would be the best way to organize
those chunks of geometry data, how would I manage engine state from
chunk to chunk? E.g. switching shaders, would switching shaders mid
render even be viable?







Currently my idea for a graphics engine involves
something like an engine class that takes render packets of data, has
some sort of ability to sort them into an order that best facilitates
speedy rendering, and changes its state depending on the information in
the packets.

This makes me think about

http://realtimecollisiondetection.net/blog/?p=86

but you probably don't mean this kind of low-level sort... or do
you? According to what you write next, it seems you're thinking at a
much higher level which apparently seems to be scenegraph's
responsability. I don't know.

As a final note here, the link I provided is effective but until
you don't finalize the systems involved it creates a strong dependancy
which will slow you down (where in the hierarchy should I put this
batch?). Start easy first! Leave the order emerge from your design
first. Benchmark. Refine. Repeat. Be aware of premature optimization.
When in doubt write easy to understand code, the bugs will generally
emerge some time later, often because of change in requirements, and you
want the code to NOT bite you. You probably have plenty of perf anyway
and your first objective is get the work done.





So I suppose what I'm looking for are book suggestions,
or perhaps open-sourced graphics engines that I could look at and learn
from ... the trouble is there aren't any such positions available where
I live, and relocation would be difficult to justify based upon the
possibly low salary expectations of a junior position

It is always a good idea to consider already available engines.
You really have to look around you (at least your state as very minimum)
and figure out what to do as there's a serious social aspect to
consider here.

From the few demos I've given out I can say that presenting
something which is 100% yours is fairly impressive, especially for small
realities, it seems that the idea of having somebody which can work out
problems top-to-bottom is very appreciated. Problem is that it takes a
lot - a lot - of effort to get to the first demo. It still takes a lot
to go to the second as you will keep working on it which in fact will
result in less demos (or reuse of the very same one).

Second problem is, as other people have said, turning a pre-made
engine into something yours is indeed valuable skill as well, which is
likely much more important if you're going to apply in consolidated
companies. Not my case here... :-(



Pre-made engines give you a great kickstart (it's awesome for
morale)! On the cons, I have to admit some projects I've looked at
seemed to be halting after a while. Let's not hide the fact: once you
choose an engine you choose a line of thinking and this implies you will
think your design in a certain way. Which is easier. The day you get a
problem which is not directly solvable by the engine, you start having
issues.

I say this because a friend of mine had exactly this problem.
Its initial enthusiasm of having "parallax mapping and shadows
everywhere" quickly faded when he figured out the hard truth: the engine
prevented him from understanding the problems involved.

Of course, if you take your time to make sure you know what
you're doing, this should not be a problem and you'll get best of both
worlds. I just wanted to drop this warning.



Your reply and other people's has given me a perspective that I didn't
consider before - taking someone elses engine and modifying it to suit
my own needs would certainly be an exemplary show of skill, especially
for studios that use 3rd party tools themselves





Experimenting a bit with modding games such as Unreal or Quake
is very useful IMHO. If you haven't tried making a Quake map then you
should really try right now. Take a look at the formats. Do some black
box analysis. Take your time... if you can afford it.

Also try to figure out what assets you have access to and make
sure you understand the data encoded in ("know your data"). I've
scratched my head more than a year in making a shader work according to
a "reference implementation" before figuring out that the "reference"
asset pipeline is way smarter than I originally believed, let's say it
works around data format's dumbness...



Actually it's funny you mention Quake mapping, when I started fiddling
with games and programming, I started with programming q-basic and
making Doom maps, which eventually progressed to Quake and Half-Life (I
rather enjoyed Worldcraft, even though it took forever and a day for
them to release decent texture mapping tools) and ignited my desire to
become a programmer (luckily I was young enough at the time that abusing
goto didn't stick with me for long). It's also why I have Michael
Abrash's Black Book, I'm fascinated by fps engines and how they
calculate potentially visible data from a vast map.







I hope this helps,

Massimo



Greatly so, thank you.




_______________________________________________
Sweng-Gamedev mailing list
Sweng-***@lists.midnightryder.com

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.
com

[Massimo Del Zotto]

2011/3/19 Nathan Martz <***@doublefine.com>
>
> but if your goal is to land a job at a high end game dev studio, there are,
> for my money, few more impressive things you could demonstrate than a
> renderer that is thoughtfully and productively parallel.
>
>
> I find this very interesting but in practice, what does that mean?
When I talk about the way I deal with shader combinatorial explosion I have
an easy way to show the benefit. I talk about the "basic" shaders
(admittedly choosen ad-hoc to maximize drama), I show some slides, I
estimate the benefit in terms of production gained.
But this is more performance oriented, more technical. Perhaps it involves
specific "performance patterns", perhaps it needs adeguate workload. If I
had to demo this thing, I wouldn't know where to start.

I'd like to read some further elaborations on that.

Thank you in advance.
Massimo

[Nathan Martz]

It could mean a variety of things. Maybe it's just that you thought
about the implications of these trends as you were writing the renderer.
That way when you get questions in your interview about how you would
handle parallelism, you'll have a thoughtful answer. Often being able to
say "well, it wasn't worth the time to implement, but I designed it make
that easy in the future because I..." is totally sufficient. If you want
something more tangible, try something like dynamic GI or one of the
more advanced types of AA which, depending on the hardware you are
targeting, might involve a fair bit of back and forth between the GPU
and the CPU (while making sure that neither is ever starved for work).



Another way to put it is that folks tend to be impressed when you solve
problems that are contemporary or even a bit future looking. A couple
decades ago, showing a demo of a bunch of bouncing env mapped sphere
rendered in OpenGL would have been pretty cutting edge. Now, it's not
quite so impressive. Similarly, shader permutations is something that
people were wrangling 5+ years ago, but is pretty sorted out now (unless
you are talking about a toolchain/workflow idea in which case, as Jon
says, could be quite cool). IMHO, most of today's interesting graphics
problems involve a lot of parallelism.



Lastly, I realized while writing this that you might get more out of
targeting interesting features (terrain rendering, realtime GI,
whatever) and over time evolving those features into a renderer than
trying to start very broadly. Again, I don't want to tell you what
project to do or where you should start, just hoping that this
perspective informs your ideas and helps you chose the most productive
course.



-Nathan



From: sweng-gamedev-***@lists.midnightryder.com
[mailto:sweng-gamedev-***@lists.midnightryder.com] On Behalf Of
Massimo Del Zotto
Sent: Sunday, March 20, 2011 3:21 AM
To: sweng-***@midnightryder.com
Subject: Re: [Sweng-Gamedev] Architecting and programming a graphics
engine in C++ & OpenGL



2011/3/19 Nathan Martz <***@doublefine.com>

but if your goal is to land a job at a high end game dev studio, there
are, for my money, few more impressive things you could demonstrate than
a renderer that is thoughtfully and productively parallel.



I find this very interesting but in practice, what does that mean?

When I talk about the way I deal with shader combinatorial explosion I
have an easy way to show the benefit. I talk about the "basic" shaders
(admittedly choosen ad-hoc to maximize drama), I show some slides, I
estimate the benefit in terms of production gained.

But this is more performance oriented, more technical. Perhaps it
involves specific "performance patterns", perhaps it needs adeguate
workload. If I had to demo this thing, I wouldn't know where to start.



I'd like to read some further elaborations on that.



Thank you in advance.

Massimo

[Jon Watte]

In this context:
I've found that engineers who want to talk and think only about the most
cutting edge shading techniques are generally not as useful to a greater
organization than an engineer who can work intelligently in this area, but
also understands the needs of a business.
For example, someone who can write a graphics engine that scales from a GMA
950 to a GTX 580 is a lot more valuable to me than someone who can shave a
HLSL instruction out from a multi-basis global illumination approximation
shader, that only will run on that GTX 580 in the first place.
Similarly, actually getting the math of graphics right is a lot more
important than any particular detail knowledge.

Here are some questions that matter a lot to me, and drive the overall
design of a rendering engine, even though they are generally derived from
the basic math of graphics:

What are the different coordinate spaces, and how do you transform between
them?
What are some examples of work that's done in these different spaces?
What's the difference between global vs local illumination, anyway?
What happens to the tangent space when you skin an animated character?
How do you blend animations without artifacts?
How do you make skinned character armpits not look like ass when applying
blended animations or rag-doll kinetics?
Which rendering techniques to make things look better that use artist
hinting do you use?
How can you make the artist hinting process as foolproof as possible?
What is an "inverse bind pose?"
Where in the content pipeline does it get applied?
If you support both vertex morphing (blend target animation) and skinning
(bone blending animation), does this change the answers?
Why is it wrong to calculate dynamic ambient occlusion in screen space? What
are some alternatives?
Why is it wrong to blur dynamic shadows in screen space? What are
some alternatives?

You'd be surprised how many "shader experts" would get half of these wrong
:-(

Separately, understanding how to break up all needed work into multiple,
independent units of work, and schedule them across available cores, is
useful, and can actually be useful in areas other than graphics, so being
able to show intelligence in this area is also good. It's quite separable
from graphics, though -- except for the detail of how APIs don't really let
you be multi-threaded (except for D3D11 and some console stuff)

Sincerely,

jw

--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.



On Sun, Mar 20, 2011 at 1:08 PM, Nathan Martz <***@doublefine.com> wrote:

> It could mean a variety of things. Maybe it’s just that you thought about
> the implications of these trends as you were writing the renderer. That way
> when you get questions in your interview about how you would handle
> parallelism, you’ll have a thoughtful answer. Often being able to say “well,
> it wasn’t worth the time to implement, but I designed it make that easy in
> the future because I
” is totally sufficient. If you want something more
> tangible, try something like dynamic GI or one of the more advanced types of
> AA which, depending on the hardware you are targeting, might involve a fair
> bit of back and forth between the GPU and the CPU (while making sure that
> neither is ever starved for work).
>
>
>
> Another way to put it is that folks tend to be impressed when you solve
> problems that are contemporary or even a bit future looking. A couple
> decades ago, showing a demo of a bunch of bouncing env mapped sphere
> rendered in OpenGL would have been pretty cutting edge. Now, it’s not quite
> so impressive. Similarly, shader permutations is something that people were
> wrangling 5+ years ago, but is pretty sorted out now (unless you are talking
> about a toolchain/workflow idea in which case, as Jon says, could be quite
> cool). IMHO, most of today’s interesting graphics problems involve a lot of
> parallelism.
>
>
>
> Lastly, I realized while writing this that you might get more out of
> targeting interesting features (terrain rendering, realtime GI, whatever)
> and over time evolving those features into a renderer than trying to start
> very broadly. Again, I don’t want to tell you what project to do or where
> you should start, just hoping that this perspective informs your ideas and
> helps you chose the most productive course.
>
>
>
> -Nathan
>
>
>
> *From:* sweng-gamedev-***@lists.midnightryder.com [mailto:
> sweng-gamedev-***@lists.midnightryder.com] *On Behalf Of *Massimo Del
> Zotto
> *Sent:* Sunday, March 20, 2011 3:21 AM
>
> *To:* sweng-***@midnightryder.com
> *Subject:* Re: [Sweng-Gamedev] Architecting and programming a graphics
> engine in C++ & OpenGL
>
>
>
> 2011/3/19 Nathan Martz <***@doublefine.com>
>
> but if your goal is to land a job at a high end game dev studio, there are,
> for my money, few more impressive things you could demonstrate than a
> renderer that is thoughtfully and productively parallel.
>
>
>
> I find this very interesting but in practice, what does that mean?
>
> When I talk about the way I deal with shader combinatorial explosion I have
> an easy way to show the benefit. I talk about the "basic" shaders
> (admittedly choosen ad-hoc to maximize drama), I show some slides, I
> estimate the benefit in terms of production gained.
>
> But this is more performance oriented, more technical. Perhaps it involves
> specific "performance patterns", perhaps it needs adeguate workload. If I
> had to demo this thing, I wouldn't know where to start.
>
>
>
> I'd like to read some further elaborations on that.
>
>
>
> Thank you in advance.
>
> Massimo
>
>
>
>
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Darren Grant]

At 02:02 PM 3/20/2011, Jon Watte wrote:
>In this context:
>I've found that engineers who want to talk and
>think only about the most cutting edge shading
>techniques are generally not as useful to a
>greater organization than an engineer who can
>work intelligently in this area, but also understands the needs of a business.
>For example, someone who can write a graphics
>engine that scales from a GMA 950 to a GTX 580
>is a lot more valuable to me than someone who
>can shave a HLSL instruction out from a
>multi-basis global illumination approximation
>shader, that only will run on that GTX 580 in the first place.
>Similarly, actually getting the math of graphics
>right is a lot more important than any particular detail knowledge.
>Â
>Here are some questions that matter a lot to me,
>and drive the overall design of a rendering
>engine, even though they are generally derived from the basic math of graphics:
>Â
>What are the different coordinate spaces, and
>how do you transform between them?
>What are some examples of work that's done in these different spaces?
>What's the difference between global vs local illumination, anyway?
>What happens to the tangent space when you skin an animated character?
>How do you blend animations without artifacts?
>How do you make skinned character armpits not
>look like ass when applying blended animations or rag-doll kinetics?
>Which rendering techniques to make things look
>better that use artist hinting do you use?
>How can you make the artist hinting process as foolproof as possible?
>What is an "inverse bind pose?"
>Where in the content pipeline does it get applied?
>If you support both vertex morphing (blend
>target animation) and skinning (bone blending
>animation), does this change the answers?
>Why is it wrong to calculate dynamic ambient
>occlusion in screen space? What are some alternatives?
>Why is it wrong to blur dynamic shadows in
>screen space? What are some alternatives?
>Â
>You'd be surprised how many "shader experts" would get half of these wrong :-(


Those are fantastic questions. It is crazy how
much domain knowledge is primarily concerned with character modelling.


Here is another tricky question that I happen to like a lot:

What is the correct way to convert features of a
scene from one coordinate system to another?


Cheers,
Darren

[Massimo Del Zotto]

Uhm, I'll definitely have to bookmark this as a future "todo" list at this
point as this makes a lot of sense. At the complexity point I have now I
haven't yet had the chance of dealing with most of those problems myself.
I'll probably want to drop a line when this happens.

Thank you for your answers.

Massimo

2011/3/20 Darren Grant <***@kerberos-productions.com>

> At 02:02 PM 3/20/2011, Jon Watte wrote:
>

[Guy Rabiller]

Hi,

It's interresting to see that most answers to a the "architecting and
programming a graphics engine in C++" question are geared toward a
"game" graphics engine.

While this is understandable, there are many forms of graphics engines.

As an exemple, take the game graphics engines on one side, and the
automotive oriented graphics engines on another side.

The first ones must be scalables, work on (relatively) old graphic
cards, must be real time and and from my point of view have this
inevitable "game look" rendering, even the lastest incarnation of some
high end graphics/game engines.

The second ones are not always realtime, rather work at interactive
rates, work on the most recent cards, but show realistic rendering.

IMHO I think the first question you have to answer is what kind of
renderer you want to achieve, and what kind of result - or visual look -
you want to achieve because this answer alone has a lot of implications
in a graphic engine design.

Visual look is not just a question of features, but a question of image
culture and understandings about what makes a picture considered as
"realistic rendering" - if it's the goal, of course.

Regarding realistic rendering, you have to work in the VFX industry to
understand how this works. Most peoples in the game industry don't
understand this, despite all the tools and features they have.

Linear workflow ( sRGB framebuffer.. ), ambient occlusion, global
illumination, HDR, etc.. all came from the VFX/CGI industry and it's
funny to see, for instance, that even nowadays, some game engines SDK (
UDK ) still invite you to use ambient occlusion with diffuse ( multiply
), wich is obviously totaly wrong. Ambient occlusion has been created to
be use with the ambient part of a lighting process wich splits ambient
lighting from diffuse lighting ( as an attempt to cheat global
illumination and it works pretty well in the VFX/CGI industry ), but
this ambient lighting is totaly absent from the graphics engine
rendering pipeline. How then it is supposed to work correctly ? It can't.

My point beeing that designing a graphics engine is one thing, it is
certainly hard, but with time it can be done, so I encourage you to go
on, but, what is perhaps more important - and I think this is where
you'll learn the most - is to understand what kind of look your renderer
will be able to achieve, and you will be able to answer this question by
first understanding *what* make a rendering having this look you want
and how to achive it ( realtime or not ).

Cheers,
Guy.
--
guy rabiller | radfac founder / ceo


Le 20/03/2011 22:02, Jon Watte a écrit :
> In this context:
> I've found that engineers who want to talk and think only about the
> most cutting edge shading techniques are generally not as useful to a
> greater organization than an engineer who can work intelligently in
> this area, but also understands the needs of a business.
> For example, someone who can write a graphics engine that scales from
> a GMA 950 to a GTX 580 is a lot more valuable to me than someone who
> can shave a HLSL instruction out from a multi-basis global
> illumination approximation shader, that only will run on that GTX 580
> in the first place.
> Similarly, actually getting the math of graphics right is a lot more
> important than any particular detail knowledge.
> Here are some questions that matter a lot to me, and drive the overall
> design of a rendering engine, even though they are generally derived
> from the basic math of graphics:
> What are the different coordinate spaces, and how do you transform
> between them?
> What are some examples of work that's done in these different spaces?
> What's the difference between global vs local illumination, anyway?
> What happens to the tangent space when you skin an animated character?
> How do you blend animations without artifacts?
> How do you make skinned character armpits not look like ass when
> applying blended animations or rag-doll kinetics?
> Which rendering techniques to make things look better that use artist
> hinting do you use?
> How can you make the artist hinting process as foolproof as possible?
> What is an "inverse bind pose?"
> Where in the content pipeline does it get applied?
> If you support both vertex morphing (blend target animation) and
> skinning (bone blending animation), does this change the answers?
> Why is it wrong to calculate dynamic ambient occlusion in screen
> space? What are some alternatives?
> Why is it wrong to blur dynamic shadows in screen space? What are
> some alternatives?
> You'd be surprised how many "shader experts" would get half of these
> wrong :-(
> Separately, understanding how to break up all needed work into
> multiple, independent units of work, and schedule them across
> available cores, is useful, and can actually be useful in areas other
> than graphics, so being able to show intelligence in this area is also
> good. It's quite separable from graphics, though -- except for the
> detail of how APIs don't really let you be multi-threaded (except for
> D3D11 and some console stuff)
> Sincerely,
> jw
>
> --
> Americans might object: there is no way we would sacrifice our living
> standards for the benefit of people in the rest of the world.
> Nevertheless, whether we get there willingly or not, we shall soon
> have lower consumption rates, because our present rates are
> unsustainable.
>
>
>
> On Sun, Mar 20, 2011 at 1:08 PM, Nathan Martz <***@doublefine.com
> <mailto:***@doublefine.com>> wrote:
>
> It could mean a variety of things. Maybe it’s just that you
> thought about the implications of these trends as you were writing
> the renderer. That way when you get questions in your interview
> about how you would handle parallelism, you’ll have a thoughtful
> answer. Often being able to say “well, it wasn’t worth the time to
> implement, but I designed it make that easy in the future because
> I
” is totally sufficient. If you want something more tangible,
> try something like dynamic GI or one of the more advanced types of
> AA which, depending on the hardware you are targeting, might
> involve a fair bit of back and forth between the GPU and the CPU
> (while making sure that neither is ever starved for work).
>
> Another way to put it is that folks tend to be impressed when you
> solve problems that are contemporary or even a bit future looking.
> A couple decades ago, showing a demo of a bunch of bouncing env
> mapped sphere rendered in OpenGL would have been pretty cutting
> edge. Now, it’s not quite so impressive. Similarly, shader
> permutations is something that people were wrangling 5+ years ago,
> but is pretty sorted out now (unless you are talking about a
> toolchain/workflow idea in which case, as Jon says, could be quite
> cool). IMHO, most of today’s interesting graphics problems involve
> a lot of parallelism.
>
> Lastly, I realized while writing this that you might get more out
> of targeting interesting features (terrain rendering, realtime GI,
> whatever) and over time evolving those features into a renderer
> than trying to start very broadly. Again, I don’t want to tell you
> what project to do or where you should start, just hoping that
> this perspective informs your ideas and helps you chose the most
> productive course.
>
> -Nathan
>
> *From:*sweng-gamedev-***@lists.midnightryder.com
> <mailto:sweng-gamedev-***@lists.midnightryder.com>
> [mailto:sweng-gamedev-***@lists.midnightryder.com
> <mailto:sweng-gamedev-***@lists.midnightryder.com>] *On Behalf
> Of *Massimo Del Zotto
> *Sent:* Sunday, March 20, 2011 3:21 AM
>
> *To:* sweng-***@midnightryder.com
> <mailto:sweng-***@midnightryder.com>
> *Subject:* Re: [Sweng-Gamedev] Architecting and programming a
> graphics engine in C++ & OpenGL
>
> 2011/3/19 Nathan Martz <***@doublefine.com
> <mailto:***@doublefine.com>>
>
> but if your goal is to land a job at a high end game dev studio,
> there are, for my money, few more impressive things you could
> demonstrate than a renderer that is thoughtfully and productively
> parallel.
>
> I find this very interesting but in practice, what does that mean?
>
> When I talk about the way I deal with shader combinatorial
> explosion I have an easy way to show the benefit. I talk about the
> "basic" shaders (admittedly choosen ad-hoc to maximize drama), I
> show some slides, I estimate the benefit in terms of production
> gained.
>
> But this is more performance oriented, more technical. Perhaps it
> involves specific "performance patterns", perhaps it needs
> adeguate workload. If I had to demo this thing, I wouldn't know
> where to start.
>
> I'd like to read some further elaborations on that.
>
> Thank you in advance.
>
> Massimo
>
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> <mailto:Sweng-***@lists.midnightryder.com>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

[Tommy Brett]

Hey Guy (and everyone else),

Thanks for that interesting read, I do have a (sort of) specific look that I
want for the engine which given my current lack of knowledge of industry
terms is a little hard to describe. When I showed my terrain renderer
prototype to a few friends on a forum I sometimes visit their first reaction
was 'but it would look great with normal mapping'. When I think of normal
maps, or bump maps, I always see the same extremely high contrasting cave
walls from Oblivion, or the lighting in Doom 3, and how there seemed to
literally be no ambient lighting at all, no attempt to even trick the player
into thinking that any sort of radiosity existed. This is something I want
to studiously avoid. I want my terrain to look real in a Half-Life 2 sort of
real, not in an almost unreal Oblivion (or in some cases the first Crysis,
with its massive over abundance of rocks and pebbles) style. Which is not to
say that I wouldn't want to use normal or bump mapping, but your post made
me consider the kinds of features I want the engine to include.

My goal before I begin writing a graphics *engine* that handles terrain is
to get my prototype looking the way I want the finished engine to look, that
way I will have learnt all the rendering methods I'll require to create the
engine. In the mean time I'm also looking forward and considering how I
might begin building the engine, hence my post to the mailing list.

Tommy

On Mon, Mar 21, 2011 at 6:16 AM, Guy Rabiller <***@grabiller.com> wrote:

> Hi,
>
> It's interresting to see that most answers to a the "architecting and
> programming a graphics engine in C++" question are geared toward a "game"
> graphics engine.
>
> While this is understandable, there are many forms of graphics engines.
>
> As an exemple, take the game graphics engines on one side, and the
> automotive oriented graphics engines on another side.
>
> The first ones must be scalables, work on (relatively) old graphic cards,
> must be real time and and from my point of view have this inevitable "game
> look" rendering, even the lastest incarnation of some high end graphics/game
> engines.
>
> The second ones are not always realtime, rather work at interactive rates,
> work on the most recent cards, but show realistic rendering.
>
> IMHO I think the first question you have to answer is what kind of renderer
> you want to achieve, and what kind of result - or visual look - you want to
> achieve because this answer alone has a lot of implications in a graphic
> engine design.
>
> Visual look is not just a question of features, but a question of image
> culture and understandings about what makes a picture considered as
> "realistic rendering" - if it's the goal, of course.
>
> Regarding realistic rendering, you have to work in the VFX industry to
> understand how this works. Most peoples in the game industry don't
> understand this, despite all the tools and features they have.
>
> Linear workflow ( sRGB framebuffer.. ), ambient occlusion, global
> illumination, HDR, etc.. all came from the VFX/CGI industry and it's funny
> to see, for instance, that even nowadays, some game engines SDK ( UDK )
> still invite you to use ambient occlusion with diffuse ( multiply ), wich is
> obviously totaly wrong. Ambient occlusion has been created to be use with
> the ambient part of a lighting process wich splits ambient lighting from
> diffuse lighting ( as an attempt to cheat global illumination and it works
> pretty well in the VFX/CGI industry ), but this ambient lighting is totaly
> absent from the graphics engine rendering pipeline. How then it is supposed
> to work correctly ? It can't.
>
> My point beeing that designing a graphics engine is one thing, it is
> certainly hard, but with time it can be done, so I encourage you to go on,
> but, what is perhaps more important - and I think this is where you'll learn
> the most - is to understand what kind of look your renderer will be able to
> achieve, and you will be able to answer this question by first understanding
> *what* make a rendering having this look you want and how to achive it (
> realtime or not ).
>
> Cheers,
> Guy.
> --
> guy rabiller | radfac founder / ceo
>
>
> Le 20/03/2011 22:02, Jon Watte a écrit :
>
> In this context:
> I've found that engineers who want to talk and think only about the most
> cutting edge shading techniques are generally not as useful to a greater
> organization than an engineer who can work intelligently in this area, but
> also understands the needs of a business.
> For example, someone who can write a graphics engine that scales from a GMA
> 950 to a GTX 580 is a lot more valuable to me than someone who can shave a
> HLSL instruction out from a multi-basis global illumination approximation
> shader, that only will run on that GTX 580 in the first place.
> Similarly, actually getting the math of graphics right is a lot more
> important than any particular detail knowledge.
>
> Here are some questions that matter a lot to me, and drive the overall
> design of a rendering engine, even though they are generally derived from
> the basic math of graphics:
>
> What are the different coordinate spaces, and how do you transform between
> them?
> What are some examples of work that's done in these different spaces?
> What's the difference between global vs local illumination, anyway?
> What happens to the tangent space when you skin an animated character?
> How do you blend animations without artifacts?
> How do you make skinned character armpits not look like ass when applying
> blended animations or rag-doll kinetics?
> Which rendering techniques to make things look better that use artist
> hinting do you use?
> How can you make the artist hinting process as foolproof as possible?
> What is an "inverse bind pose?"
> Where in the content pipeline does it get applied?
> If you support both vertex morphing (blend target animation) and skinning
> (bone blending animation), does this change the answers?
> Why is it wrong to calculate dynamic ambient occlusion in screen space?
> What are some alternatives?
> Why is it wrong to blur dynamic shadows in screen space? What are
> some alternatives?
>
> You'd be surprised how many "shader experts" would get half of these wrong
> :-(
>
> Separately, understanding how to break up all needed work into multiple,
> independent units of work, and schedule them across available cores, is
> useful, and can actually be useful in areas other than graphics, so being
> able to show intelligence in this area is also good. It's quite separable
> from graphics, though -- except for the detail of how APIs don't really let
> you be multi-threaded (except for D3D11 and some console stuff)
>
> Sincerely,
>
> jw
>
> --
> Americans might object: there is no way we would sacrifice our living
> standards for the benefit of people in the rest of the world. Nevertheless,
> whether we get there willingly or not, we shall soon have lower consumption
> rates, because our present rates are unsustainable.
>
>
>
> On Sun, Mar 20, 2011 at 1:08 PM, Nathan Martz <***@doublefine.com>wrote:
>
>> It could mean a variety of things. Maybe it’s just that you thought
>> about the implications of these trends as you were writing the renderer.
>> That way when you get questions in your interview about how you would handle
>> parallelism, you’ll have a thoughtful answer. Often being able to say “well,
>> it wasn’t worth the time to implement, but I designed it make that easy in
>> the future because I…” is totally sufficient. If you want something more
>> tangible, try something like dynamic GI or one of the more advanced types of
>> AA which, depending on the hardware you are targeting, might involve a fair
>> bit of back and forth between the GPU and the CPU (while making sure that
>> neither is ever starved for work).
>>
>>
>>
>> Another way to put it is that folks tend to be impressed when you solve
>> problems that are contemporary or even a bit future looking. A couple
>> decades ago, showing a demo of a bunch of bouncing env mapped sphere
>> rendered in OpenGL would have been pretty cutting edge. Now, it’s not quite
>> so impressive. Similarly, shader permutations is something that people were
>> wrangling 5+ years ago, but is pretty sorted out now (unless you are talking
>> about a toolchain/workflow idea in which case, as Jon says, could be quite
>> cool). IMHO, most of today’s interesting graphics problems involve a lot of
>> parallelism.
>>
>>
>>
>> Lastly, I realized while writing this that you might get more out of
>> targeting interesting features (terrain rendering, realtime GI, whatever)
>> and over time evolving those features into a renderer than trying to start
>> very broadly. Again, I don’t want to tell you what project to do or where
>> you should start, just hoping that this perspective informs your ideas and
>> helps you chose the most productive course.
>>
>>
>>
>> -Nathan
>>
>>
>>
>> *From:* sweng-gamedev-***@lists.midnightryder.com [mailto:
>> sweng-gamedev-***@lists.midnightryder.com] *On Behalf Of *Massimo Del
>> Zotto
>> *Sent:* Sunday, March 20, 2011 3:21 AM
>>
>> *To:* sweng-***@midnightryder.com
>> *Subject:* Re: [Sweng-Gamedev] Architecting and programming a graphics
>> engine in C++ & OpenGL
>>
>>
>>
>> 2011/3/19 Nathan Martz <***@doublefine.com>
>>
>> but if your goal is to land a job at a high end game dev studio, there
>> are, for my money, few more impressive things you could demonstrate than a
>> renderer that is thoughtfully and productively parallel.
>>
>>
>>
>> I find this very interesting but in practice, what does that mean?
>>
>> When I talk about the way I deal with shader combinatorial explosion I
>> have an easy way to show the benefit. I talk about the "basic" shaders
>> (admittedly choosen ad-hoc to maximize drama), I show some slides, I
>> estimate the benefit in terms of production gained.
>>
>> But this is more performance oriented, more technical. Perhaps it involves
>> specific "performance patterns", perhaps it needs adeguate workload. If I
>> had to demo this thing, I wouldn't know where to start.
>>
>>
>>
>> I'd like to read some further elaborations on that.
>>
>>
>>
>> Thank you in advance.
>>
>> Massimo
>>
>>
>>
>>
>>
>> _______________________________________________
>> Sweng-Gamedev mailing list
>> Sweng-***@lists.midnightryder.com
>>
>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>
>>
>
> _______________________________________________
> Sweng-Gamedev mailing listSweng-***@lists.midnightryder.comhttp://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>

[Jon Watte]

I would recommend anything by Eric Lengyel, and whatever the latest
iteration is of David Eberly's set of books. Neither is perfect, but both
are written by people who actually have clues (which is somewhat rare in the
tutorials area, I've found :-)

The other thing I would recommend is to spend significant time trying to
develop a game -- any game -- within a number of different game engines.
Panda3D. C4 engine. Torque (if it's still around). UDK. Unity. The list goes
on, but if you try to develop a game idea/concept with each of those tools,
far enough that you actually get to something playable in each, then you
should have learned a number of different ways to do things.

Also, it's important to understand the art creation part of the graphics
pipeline. Learning at least one of the top modeling packages (Max, Maya,
etc) is highly useful, and poking at a variety of art pipelines (Colllada,
Havok, Granny, etc) is also helpful. Most game engines mentioned also come
with art pipelines, but somehow, each of them comes up short in at least one
area -- this is also an important lesson!

No reading can replace the practice of actually doing things, and because
you need to spend a month or two with significant effort on each engine, you
probably won't have the breadth of experience needed completed this year,
but it's an approach I highly recommend!

Good luck, and I hope this helps.

Sincerely,

jw

--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.



On Fri, Mar 18, 2011 at 5:52 AM, Tommy Brett <***@gmail.com> wrote:

> Hi folks,
>
> I've been working as an AS3 programmer / animator for the last 3.5 years,
> and have the strongest desire to leave the marketing industry behind and
> pursue my original intended career as a games programmer, which I sidelined
> due to a very coincidental series of events. To this end, I've been (slowly)
> building a graphics engine using C++, OpenGL and nVidia CG which renders
> very large chunks of terrain using GPU Clipmaps. The idea is that eventually
> it will render planets with atmosphere and real time shadows, or die trying,
> and in any case will serve as my main portfolio piece when I eventually
> start applying for positions. Because I've been outside of my programming
> comfort zone for so long it's been slow going, but now that I've finally got
> a firm grasp on shaders and the OpenGL pipeline it's time to look to the
> future.
>
> This is where my problem lies; although there are many examples on the 'net
> showing me how to animate the most beautiful normal mapped cobbled cube, or
> tutorials on specific rendering / culling / lighting algorithms, I've yet to
> find a resource beside Michael Abrash's Graphics Programming Black Book
> (which sits on my desk, reminding me of what I should have been doing all
> along) which goes into the subject of writing a graphics engine. See
> although I can write perfectly passable C++, and thanks to my AS3
> programming experience which has frequently required that I get my projects
> right 'the first time', I'm fairly adept at planning and designing my
> classes before I begin... But from a C++ standpoint, I'm somewhat in the
> dark. What are the best practices for building a graphics engine in C++?
> Exactly how OOP should I go? How should I handle the use of multiple shaders
> on chunks of geometry? There's just this giant multitude of questions, and
> although I have a knack for over engineering things, I really want to have a
> better picture of what I'm aiming for before I start, or I'll end up with
> the monstrosity that is my terrain prototype (it runs, but it's so, so
> ugly).
>
> Currently my idea for a graphics engine involves something like an engine
> class that takes render packets of data, has some sort of ability to sort
> them into an order that best facilitates speedy rendering, and changes its
> state depending on the information in the packets. E.g. one packet might be
> a chunk of terrain, and could include information about what shaders to use,
> how to determine its potential visible set, and the type of collision
> detection to use (though this deviates a little from the graphics engine
> part). There might be an arbitrary number of terrain packets, followed by
> some character models or buildings using a different type of culling
> algorithm. My worry aside from the obvious problem of the whole idea
> possibly being crap, is that it's too general. How could I possibly create
> an engine that handles every possible rendering type, shouldn't I perhaps
> start with the assertion that I'm building a terrain engine, and base
> everything around rendering sphere like chunks of geometry with varying
> levels of detail? But on the same token, I don't just want to end up with
> another terrain engine prototype as seen on youtube(tm).
>
> So I suppose what I'm looking for are book suggestions, or perhaps
> open-sourced graphics engines that I could look at and learn from, or maybe
> I should just try and get a position as a junior-something-programmer and
> learn by doing (the trouble is there aren't any such positions available
> where I live, and relocation would be difficult to justify based upon the
> possibly low salary expectations of a junior position, I'd like at least
> some leverage). Ideas? Suggestions? Is this even the right place to ask such
> broad questions? I'm all ears.
>
> - Tommy
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-***@lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>