Sweng-Gamedev Digest, Vol 71, Issue 2

Gabriel Sassone

2013-03-14 21:27:43 UTC

Thanks for the reply,
mine is just a brain dumped exploration of how to have fast iteration
times.

It is not only an artist need to swap/add assets, but my general feeling is
about other programming workflows that can be used.
Like for gameplay programmers, having a runtime compiled c++ could increase
a lot iterations and ideas.

What I don't like is this delay between coding and testing something, even
a small change...

I am not searching for patterns but for experiences!

Thanks Tinco, your is a good idea, even if ambitious! Do you have any
experience on that?
Still not speaking about work here, but more about inner-thoughts.

I feel that a reflection using clang/pdbs is far easier and use already
existing informations stored, and it makes more sense to me, in both large
or small codebases.
But I am worried about the parsing speed of the db and that you need to do
after every compiling.

On Thu, Mar 14, 2013 at 1:43 PM, <

Send Sweng-Gamedev mailing list submissions to
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
or, via email, send a message with subject or body 'help' to
You can reach the person managing the list at
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Sweng-Gamedev digest..."
1. Reflection in game engine (c++) (Gabriel Sassone)
2. Re: Reflection in game engine (c++) (Tinco Andringa)
3. Re: Reflection in game engine (c++) (Tinco Andringa)
4. Re: Reflection in game engine (c++) (Thatcher Ulrich)
5. Re: Reflection in game engine (c++) (Thatcher Ulrich)
6. Re: Reflection in game engine (c++) (Tinco Andringa)
----------------------------------------------------------------------
Message: 1
Date: Thu, 14 Mar 2013 11:09:30 -0700
Subject: [Sweng-Gamedev] Reflection in game engine (c++)
<CAFcPrsFBD6mgMFR8dFsi3=
Content-Type: text/plain; charset="iso-8859-1"
Hello gents,
I wanted to experiment with reflection and I found different way to
1) Invasive macro based registration
2) Reflection visitor pattern
3) Parsing pdb/clang stuff and create a manual table of what you
want/need
4) c++ custom parser that uses tags to create reflection informations
(like a comment near a member, ...) (exuberant ctags???)
5) create serializable/reflected classes in a data format, then run a
tool that generates headers and cpps
I am trying to figure out flaws and merits of each kind, and even if there
are other solutions.
I love the idea to have the possibility to serialize in and out structures,
access fields and change values, stream in/out stuff from network ,and
maybe link with scripting.
My goal is to have fast iteration times, both artists and programmers, and
I achieved it for rendering programmers already with having json binarized
configurable rendering, but still there are code stuff that are not easy to
achieve.
Here are my thoughts, but I would like to hear your ideas and experiences!
1) Very precise on what you can register/serialize, but painful to maintain
or plug into an existing codebase
2) Less precise but you just need one method per class, that you can use to
serialize/reflect. Still bad in maintaining, but better in plugging in.
3) You make the compiler do the work, then translate the informations you
need in your format. Easy to maintain, to plug...maybe slow in parsing?
Slow your build pipeline?
4) maybe easier than using pdbs, but less powerful (you cannot invoke
methods on objects). should be faster than 3.
5) crazy idea. requires generation of code from outside...powerful like 4,
so data reflection, but you cannot invoke methods if you want.
What are your experiences/thoughts?
Thanks to everyone that read this mail!
Gabriel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.midnightryder.com/private.cgi/sweng-gamedev-midnightryder.com/attachments/20130314/c4522aed/attachment-0001.htm
------------------------------
Message: 2
Date: Thu, 14 Mar 2013 19:36:21 +0100
Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)
<CAGW=
Content-Type: text/plain; charset=UTF-8
Hi Gabriel,
In an effort to make C/C++ more like C# I did a bit of research into
adding type/signature information to binaries using clang. A downside
of this approach is that you're restricted to the clang compiler, but
in return you get a lot of power over the compilation process.
The idea is rather simple, the compiler visits all nodes in the C++
AST, you can write a clang module that hooks into that process and
record type information and then at the end store it in a section of
the binary. Then in your code you can simply refer to that section and
read it to get type information.
But that being said, it's a rather ambitious project and I don't
really see how going on such a quest is a wise time investment for
your game pipeline.
Some of the things you mention (serializing, streaming) can be done by
having a neat data model and a good serialization protocol. For
example if your game uses a CES architecture you just need a
serialization for the data structure of each component. Together with
the id's of the components and the objects you could stream in or out
any aspect of the game without ever needing reflection.
I imagine you could even dynamically load/replace components sent over
the network, enabling some sort of hot code swapping :)
Is your game architected in such a way that that would be possible?
Perhaps if your codebase is very large already it's not feasible to
rearchitect to such an extent, though I've heard some studios switched
to CES in the middle of their dev process. It is possible to gradually
migrate from hierarchical architecture to CES (I'm doing so for my own
project atm too).
Hope this is useful.
Kind regards,
Tinco

Hello gents,
I wanted to experiment with reflection and I found different way to
1) Invasive macro based registration
2) Reflection visitor pattern
3) Parsing pdb/clang stuff and create a manual table of what you

want/need

4) c++ custom parser that uses tags to create reflection informations
(like a comment near a member, ...) (exuberant ctags???)
5) create serializable/reflected classes in a data format, then run a

tool

that generates headers and cpps
I am trying to figure out flaws and merits of each kind, and even if

there

are other solutions.
I love the idea to have the possibility to serialize in and out

structures,

access fields and change values, stream in/out stuff from network ,and

maybe

link with scripting.
My goal is to have fast iteration times, both artists and programmers,

and I

achieved it for rendering programmers already with having json binarized
configurable rendering, but still there are code stuff that are not easy

achieve.
Here are my thoughts, but I would like to hear your ideas and

experiences!

1) Very precise on what you can register/serialize, but painful to

maintain

or plug into an existing codebase
2) Less precise but you just need one method per class, that you can use

serialize/reflect. Still bad in maintaining, but better in plugging in.
3) You make the compiler do the work, then translate the informations you
need in your format. Easy to maintain, to plug...maybe slow in parsing?

Slow

your build pipeline?
4) maybe easier than using pdbs, but less powerful (you cannot invoke
methods on objects). should be faster than 3.
5) crazy idea. requires generation of code from outside...powerful like

so data reflection, but you cannot invoke methods if you want.
What are your experiences/thoughts?
Thanks to everyone that read this mail!
Gabriel
_______________________________________________
Sweng-Gamedev mailing list

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
------------------------------
Message: 3
Date: Thu, 14 Mar 2013 19:36:21 +0100
Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)
<CAGW=
Content-Type: text/plain; charset=UTF-8
Hi Gabriel,
In an effort to make C/C++ more like C# I did a bit of research into
adding type/signature information to binaries using clang. A downside
of this approach is that you're restricted to the clang compiler, but
in return you get a lot of power over the compilation process.
The idea is rather simple, the compiler visits all nodes in the C++
AST, you can write a clang module that hooks into that process and
record type information and then at the end store it in a section of
the binary. Then in your code you can simply refer to that section and
read it to get type information.
But that being said, it's a rather ambitious project and I don't
really see how going on such a quest is a wise time investment for
your game pipeline.
Some of the things you mention (serializing, streaming) can be done by
having a neat data model and a good serialization protocol. For
example if your game uses a CES architecture you just need a
serialization for the data structure of each component. Together with
the id's of the components and the objects you could stream in or out
any aspect of the game without ever needing reflection.
I imagine you could even dynamically load/replace components sent over
the network, enabling some sort of hot code swapping :)
Is your game architected in such a way that that would be possible?
Perhaps if your codebase is very large already it's not feasible to
rearchitect to such an extent, though I've heard some studios switched
to CES in the middle of their dev process. It is possible to gradually
migrate from hierarchical architecture to CES (I'm doing so for my own
project atm too).
Hope this is useful.
Kind regards,
Tinco

want/need

4) c++ custom parser that uses tags to create reflection informations
(like a comment near a member, ...) (exuberant ctags???)
5) create serializable/reflected classes in a data format, then run a

tool

that generates headers and cpps
I am trying to figure out flaws and merits of each kind, and even if

there

are other solutions.
I love the idea to have the possibility to serialize in and out

structures,

access fields and change values, stream in/out stuff from network ,and

maybe

link with scripting.
My goal is to have fast iteration times, both artists and programmers,

and I

achieved it for rendering programmers already with having json binarized
configurable rendering, but still there are code stuff that are not easy

achieve.
Here are my thoughts, but I would like to hear your ideas and

experiences!

1) Very precise on what you can register/serialize, but painful to

maintain

or plug into an existing codebase
2) Less precise but you just need one method per class, that you can use

Slow

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
------------------------------
Message: 4
Date: Thu, 14 Mar 2013 14:55:28 -0400
Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)
<CAMa9PtQJhRwhx_tcpUOX=
Content-Type: text/plain; charset=ISO-8859-1
"CES"?

want/need

4) c++ custom parser that uses tags to create reflection informations
(like a comment near a member, ...) (exuberant ctags???)
5) create serializable/reflected classes in a data format, then run a

tool

that generates headers and cpps
I am trying to figure out flaws and merits of each kind, and even if

there

are other solutions.
I love the idea to have the possibility to serialize in and out

structures,

access fields and change values, stream in/out stuff from network ,and

maybe

link with scripting.
My goal is to have fast iteration times, both artists and programmers,

and I

achieved it for rendering programmers already with having json binarized
configurable rendering, but still there are code stuff that are not easy

achieve.
Here are my thoughts, but I would like to hear your ideas and

experiences!

1) Very precise on what you can register/serialize, but painful to

maintain

or plug into an existing codebase
2) Less precise but you just need one method per class, that you can use

Slow

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
------------------------------
Message: 5
Date: Thu, 14 Mar 2013 14:55:28 -0400
Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)
<CAMa9PtQJhRwhx_tcpUOX=
Content-Type: text/plain; charset=ISO-8859-1
"CES"?

want/need

4) c++ custom parser that uses tags to create reflection informations
(like a comment near a member, ...) (exuberant ctags???)
5) create serializable/reflected classes in a data format, then run a

tool

that generates headers and cpps
I am trying to figure out flaws and merits of each kind, and even if

there

are other solutions.
I love the idea to have the possibility to serialize in and out

structures,

access fields and change values, stream in/out stuff from network ,and

maybe

link with scripting.
My goal is to have fast iteration times, both artists and programmers,

and I

achieved it for rendering programmers already with having json binarized
configurable rendering, but still there are code stuff that are not easy

achieve.
Here are my thoughts, but I would like to hear your ideas and

experiences!

1) Very precise on what you can register/serialize, but painful to

maintain

or plug into an existing codebase
2) Less precise but you just need one method per class, that you can use

Slow

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
------------------------------
Message: 6
Date: Thu, 14 Mar 2013 21:34:15 +0100
Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)
<CAGW=
Content-Type: text/plain; charset=UTF-8
Sorry, Component Entity System, it's a way of organizing your game
engine that has some popularity amongst in the industry. Here's a
http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy/
I just read Gabriel's linkedin and he's got years of experience on big
games so I guess he really has a good reason to check if reflection
can optimize his pipeline.

"CES"?

want/need

4) c++ custom parser that uses tags to create reflection informations
(like a comment near a member, ...) (exuberant ctags???)
5) create serializable/reflected classes in a data format, then run a

tool

that generates headers and cpps
I am trying to figure out flaws and merits of each kind, and even if

there

are other solutions.
I love the idea to have the possibility to serialize in and out

structures,

access fields and change values, stream in/out stuff from network ,and

maybe

link with scripting.
My goal is to have fast iteration times, both artists and programmers,

and I

achieved it for rendering programmers already with having json binarized
configurable rendering, but still there are code stuff that are not

easy to

achieve.
Here are my thoughts, but I would like to hear your ideas and

experiences!

1) Very precise on what you can register/serialize, but painful to

maintain

or plug into an existing codebase
2) Less precise but you just need one method per class, that you can

use to

serialize/reflect. Still bad in maintaining, but better in plugging in.
3) You make the compiler do the work, then translate the informations

you

need in your format. Easy to maintain, to plug...maybe slow in parsing?

Slow

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

_______________________________________________
Sweng-Gamedev mailing list

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
------------------------------
_______________________________________________
Sweng-Gamedev mailing list
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
End of Sweng-Gamedev Digest, Vol 71, Issue 2
********************************************