On the benefits of reflection

[Massimo Del Zotto]

Hello,
this question spawned from the recent discussion on reflection. I guessed
it was better to open an independent discussion instead of hijacking the
thread.

So if you do start wandering into the land of making your own language for
just reflection, you might see what other benefits you can get from doing
more in it than just that.

I'd be interested in knowing more about doing the other thing around.
I have a small little language I've been using for a while and I've been
fairly happy with it. So far, the language itself does not have reflective
constructs (nor I plan to add them) but I use reflection internally mainly
to simulate duck typing.
So far, I haven't well understood the benefits of reflection. I had seen
some examples in ObjC but they didn't really appeared to be buying much
from me. I know reflection is there in a Java package, but I couldn't quite
figure out the whole picture.

Could you please elaborate on the benefits of using reflection? Please be
specific on whatever you're talking about reflection from the native core
or reflection from the language.

Thank you
Massimo

[Marc Hernandez]

Man, I love reflection. It allows me to just belt out working systems with
ease.

So, Im using it for:
x) Trivial network system. All my network packets are just classes that I
marked [Serializable]. If I have any special handling I use custom field
attributes to mark it up.
x) Config system. Uses all the same hardware as the networking system, now
its just loading to and from disk.
x) Save/Load. Same.
x) Message passing system. This is essentially just the networking system.
y) I have a friend that built a nice level editor in XNA in a month which
heavily relied on reflection.

With all of these I have 1 reflection handling class. Currently its a
custom written XML reader/writer that slightly abuses the XML rules so
files are small. I also have a binary reader and writer. The networking
system can negotiate for which one to use. Making a viewer looks like:
structure = Read( BinaryFormatter );
Print( XMLFormatter );

I then use reflection to automatically type safe handle the messages.
So, you just put:
void handl( msg.MoveTo move )

And higher level code calls down into it. If this shows up in profiling,
ill special case the heavily called messages and leave the rest.

Post by Massimo Del Zotto
Hello,
this question spawned from the recent discussion on reflection. I guessed
it was better to open an independent discussion instead of hijacking the
thread.

So if you do start wandering into the land of making your own language
for just reflection, you might see what other benefits you can get from
doing more in it than just that.

I'd be interested in knowing more about doing the other thing around.
I have a small little language I've been using for a while and I've been
fairly happy with it. So far, the language itself does not have reflective
constructs (nor I plan to add them) but I use reflection internally mainly
to simulate duck typing.
So far, I haven't well understood the benefits of reflection. I had seen
some examples in ObjC but they didn't really appeared to be buying much
from me. I know reflection is there in a Java package, but I couldn't quite
figure out the whole picture.
Could you please elaborate on the benefits of using reflection? Please be
specific on whatever you're talking about reflection from the native core
or reflection from the language.
Thank you
Massimo
_______________________________________________
Sweng-Gamedev mailing list
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

[Massimo Del Zotto]

Let me elaborate on those points so I hope people can get in my point of
view.

1- "Trivial" network system. Done for ages, before reflection was in vogue.
Considering how UDK manages that thing, I believe we're cutting a lot of
corners. What's the problem in just doing serialization in the standard way?
2- Config system. No idea what "the same hardware" is supposed to mean. I'd
be careful in making my networking subsystem work the same as a disk IO.
3- save load. In my system, just restoring VM state is going nowhere. Maybe
the scripts will just churn along. Sure engine state isn't going to make it
there.
4- Message passing... through the networking system? I don't see the
connection with reflection. I'd be extra super careful in doing that.
Message passing to who? For what purpose? At what frequency?
5- I am using reflection myself in my system as well. Good for your friend.
Perhaps you could contact him and ask what exactly made his work easier?
I'd be interested in considering those cases. As I said, I have this little
language of mine I want to evolve in the right direction.

I then use reflection to automatically type safe handle the messages.

Post by Marc Hernandez
void handl( msg.MoveTo move )
And higher level code calls down into it. If this shows up in
profiling, ill special case the heavily called messages and leave the rest.

I have no idea on what you mean there. I actually don't even understand
what's really the point. The prototype you show leaves me confused. If the
above is the equivalent of void handl(Message *something) then I'm afraid
this could work with dynamic_cast in some cases or with structures designed
to be somehow analyzed, perhaps through a "property" system. What is the
extra plus reflection gives me?

Massimo

[Marc Hernandez]

Post by Massimo Del Zotto
Let me elaborate on those points so I hope people can get in my point of
view.
1- "Trivial" network system. Done for ages, before reflection was in
vogue. Considering how UDK manages that thing, I believe we're cutting a
lot of corners. What's the problem in just doing serialization in the
standard way?

I havent seen a non reflection based network system that was as easy as a
reflection based one. By serialization, if you mean:
send()
health >> out;
pos >> out;

recv()
health << in;
pos << in;

A reflection system merely automates this and makes it very difficult to
get wrong even with rapid development. Its a completely DRY (dont repeat
yourself (haha)) architecture. With your traditional serialization you
have to touch 3 places in code when you change some structure, whereas with
reflection I only change one. If you want to make it simpler it just
starts getting into adhoc reflection systems.

Post by Massimo Del Zotto
2- Config system. No idea what "the same hardware" is supposed to mean.
I'd be careful in making my networking subsystem work the same as a disk IO.

Sorry, what I meant was my config system uses many of the same classes as
the networking layer. The only real difference is config uses my XML
read/write and goes to and from disk while my network does binary
read/write and goes to stream. I also save off the binary network stream
for trivial debugging later via read(binary) -> write( xml ). All from
fairly short classes I wrote once.

Post by Massimo Del Zotto
3- save load. In my system, just restoring VM state is going nowhere.
Maybe the scripts will just churn along. Sure engine state isn't going to
make it there.

I dont know what you mean here. Most games need a way to save out their
game state, then when the player comes back, to read it back in. I do all
that with the same reflection system that reads and writes game state,
marked up if necessary.

Post by Massimo Del Zotto
4- Message passing... through the networking system? I don't see the
connection with reflection. I'd be extra super careful in doing that.
Message passing to who? For what purpose? At what frequency?

For this I primarily mean find methods based on the signature of the
method like my handle( msg.MoveTo move ) example below. Though, I have
also used it to spread workloads across multiple machines without the
underlying work classes knowing I did.

Post by Massimo Del Zotto
5- I am using reflection myself in my system as well. Good for your
friend. Perhaps you could contact him and ask what exactly made his work
easier? I'd be interested in considering those cases. As I said, I have
this little language of mine I want to evolve in the right direction.

What made it fast for him was this. To make a new enemy, you make the
class and markup the variables. Then with absolutely no other work, you
can load up the editor. Using C# reflection, it autopopulates the enemy
list with this new enemy, and allows it to be placed. You can then place
it and fill in any members that you need to for that new guy.
Minimizing the friction of doing this is always helpful, and with
reflection you can keep code very very DRY.

I then use reflection to automatically type safe handle the messages.

Post by Massimo Del Zotto

Post by Marc Hernandez
void handl( msg.MoveTo move )
And higher level code calls down into it. If this shows up in
profiling, ill special case the heavily called messages and leave the rest.

I have no idea on what you mean there. I actually don't even understand
what's really the point. The prototype you show leaves me confused. If the
above is the equivalent of void handl(Message *something) then I'm afraid
this could work with dynamic_cast in some cases or with structures designed
to be somehow analyzed, perhaps through a "property" system. What is the
extra plus reflection gives me?

What reflection gave me is a typesafe, prewritten version of all of that
which makes me more productive. I wrote the dispatch function once, and
now I just add a handler and poof, that message is handled correctly. I
dont need to check for things, or cast, or assert or anything, it all just
works by adding 1 function.

--
/// //

[Douglas Cox]

I'm not entirely sure what your question is (perhaps an example of what
you're trying to ask may help), but I can tell you what we would do and
what the benefits are. This example is not real code, but it shows

class Texture
{
[EditableFilename( "png" )]
String filename;

[Editable( onChanged=OnFormatChanged )]
TextureFormat format;

[EditableFloat( 0.0, 1.0 )]
float scale;

TextureData data;

void OnFormatChanged()
{
... re-cache texture on PC ...
}
}

// Just some example of how we access the type info.
void DoSomethingWithFields( Object object )
{
foreach( var field in typeof( object ).fields )
Console.WriteLine( "Field: {0} {1} = {2}", field.type.name,
field.name, field.GetValue( object ) );
}

So from reflection we're obviously able to enumerate the fields of any
type, get the type of field, its attributes, and from reflection and
invoking, we can modify the field and notify the class of changes from
editor-side only if needed. Those attributes and notification methods can
easily be compiled away in final builds if need be.

Reflection can tell us what needs to save to disk or what needs to be
serialized over the network. Sure you can write your own Read/Write code
(and we did in the past), but that gets messy and is just another copy of
the information (fields) that you've already entered somewhere else. And
sure, you could have a compiler generate a Read/Write function for you, but
changing/updating that code then requires changing the compiler and
recompiling your entire code base.

Our goal is to make writing code as simple and clean as possible while
avoiding redundant typing. If you have another way to do this, then that's
cool too - I'd love to see an example.

And for us, there's also a small amount of reflection needed for our
garbage collector...

-Doug

Post by Massimo Del Zotto
Let me elaborate on those points so I hope people can get in my point of
view.
1- "Trivial" network system. Done for ages, before reflection was in
vogue. Considering how UDK manages that thing, I believe we're cutting a
lot of corners. What's the problem in just doing serialization in the
standard way?
2- Config system. No idea what "the same hardware" is supposed to mean.
I'd be careful in making my networking subsystem work the same as a disk IO.
3- save load. In my system, just restoring VM state is going nowhere.
Maybe the scripts will just churn along. Sure engine state isn't going to
make it there.
4- Message passing... through the networking system? I don't see the
connection with reflection. I'd be extra super careful in doing that.
Message passing to who? For what purpose? At what frequency?
5- I am using reflection myself in my system as well. Good for your
friend. Perhaps you could contact him and ask what exactly made his work
easier? I'd be interested in considering those cases. As I said, I have
this little language of mine I want to evolve in the right direction.
I then use reflection to automatically type safe handle the messages.

Post by Marc Hernandez
void handl( msg.MoveTo move )
And higher level code calls down into it. If this shows up in
profiling, ill special case the heavily called messages and leave the rest.

I have no idea on what you mean there. I actually don't even understand
what's really the point. The prototype you show leaves me confused. If the
above is the equivalent of void handl(Message *something) then I'm afraid
this could work with dynamic_cast in some cases or with structures designed
to be somehow analyzed, perhaps through a "property" system. What is the
extra plus reflection gives me?
Massimo
_______________________________________________
Sweng-Gamedev mailing list
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

[Tinco Andringa]

And for us, there's also a small amount of reflection needed for our garbage
collector...
-Doug

You use reflection to implement your own garbage collector? This is C#
right? Does your GC just collect resources that aren't needed anymore
or does it actually do a better job of managing memory etc?

To add to the discussion about benefits of reflection:

Coming from Ruby I'm always looking for ways to make things easier to
express using reflection. I think serialization/deserialization is
just one usecase for this. For example C# is a language that's more
geared towards object oriented architecture than to entity systems,
but using reflection you can generate the code that links your
components together at runtime so there's not a lot of boilerplate
code required for every component.

Sort of related to reflection, recently I made a game as an academic
exercise in javascript using an aspect oriented programming library.
AOP has a similar philosophy of taking separation of concerns to the
next level. It worked out very well, you can have the game engine
drive the views without having any hooks or view related code in the
engine. Granted ofcourse this was a javascript exercise, not at all as
resource constrained as a big console game.

[Douglas Cox]

Post by Tinco Andringa
You use reflection to implement your own garbage collector? This is C#
right? Does your GC just collect resources that aren't needed anymore
or does it actually do a better job of managing memory etc?

That was not exactly C# despite what it looks like -- it's our own variant
and compiler. The GC uses minimal reflection in the sense that our type
information includes a small bit of information (provided by the compiler)
useful for tracing through object references. It's a multi-generational
GC, a bit tailored to our needs. It supports finalizers, deals with
multiple threads, etc. It is used pretty much like the one in C#.

Honestly, after 12 years we just got tired of C++'s issues (see this thread
for a good example), and so far love how clean it is to write the new code.
When we need to write some C/C++ code it's as easy as:

void Foo()
{>
// c++ code goes here.
<}

It may sound like a pain to get working, but if you look at some of
runtime-recompile work, and then realize that most of the annoying parts of
that can also be hidden behind a 'pre-compiler', it isn't as bad as it
sounds. Especially when you know the data layout of your own vtables and
can run through all allocations fixing up pointers, etc.

Post by Tinco Andringa
Coming from Ruby I'm always looking for ways to make things easier to
express using reflection. I think serialization/deserialization is
just one usecase for this. For example C# is a language that's more
geared towards object oriented architecture than to entity systems,
but using reflection you can generate the code that links your
components together at runtime so there's not a lot of boilerplate
code required for every component.

Yup.

Sort of related to reflection, recently I made a game as an academic

Post by Tinco Andringa
exercise in javascript using an aspect oriented programming library.
AOP has a similar philosophy of taking separation of concerns to the
next level. It worked out very well, you can have the game engine
drive the views without having any hooks or view related code in the
engine. Granted ofcourse this was a javascript exercise, not at all as
resource constrained as a big console game.

Sounds cool. Do you have some specific examples?

-Doug

[Tinco Andringa]

Post by Douglas Cox

Post by Tinco Andringa
Sort of related to reflection, recently I made a game as an academic
exercise in javascript using an aspect oriented programming library.
AOP has a similar philosophy of taking separation of concerns to the
next level. It worked out very well, you can have the game engine
drive the views without having any hooks or view related code in the
engine. Granted ofcourse this was a javascript exercise, not at all as
resource constrained as a big console game.

Sounds cool. Do you have some specific examples?

So it kind of works sort of like an inverted events system. For
example using events:

class Object {
TakeDamage(int damage) {
var total = max(armor - damage, 0);
hp -= total;
Events.Trigger(new DamageTakenEvent(this, damage, total));
}
}

Events.On<DamageTakenEvent>( e =>
Console.WriteLine(e.Object.Name + " took: " + e.total + " damage.");
);

Using aspect oriented programming:

class Object {
TakeDamage(int damage) {
var total = max(armor - damage, 0);
hp -= total;
}
}

AOP.Around(Object.TakeDamage, (invocation, params, object) =>
var total = object.HP;
invocation.proceed(params);
total -= object.HP;
Console.WriteLine(object.Name + " took: " + total + " damage.");
);

Note that I don't really know what C# aop looks like, I'm not sure you
can be that succinct when using method references (delegations) but
that's the gist of it. You can see now the game logic function
TakeDamage has no awareness of any observers, so there's a clean
separation of concerns. But also notice that the AOP advice can not
see what happens in the method so has to recalculate the total.

In my own 'big' game I'm using the events approach btw, I'm not fully
convinced AOP is the way to go, but it is interesting.

[Massimo Del Zotto]

My question regards the goal I'm targeting. When something is worth, I can
perceive a strong set of goals "opening" at a lower effort. I don't have
that feel with reflection. I am not really able to pull out the exact
question synthethizing my doubts but I hope I will be able to explain
myself with various comments.

So from reflection we're obviously able to enumerate the fields of any
type, get the type of field, its attributes, and from reflection and
invoking, we can modify the field and notify the class of changes from
editor-side only if needed. Those attributes and notification methods can
easily be compiled away in final builds if need be.

The texture example does not fit my likings at all, perhaps I'm stuck in a
old-school reflection line of thinking but it looks to me it would be much
nicer using a component based system. Do component based systems qualify as
reflective systems?

Reflection can tell us what needs to save to disk or what needs to be
serialized over the network. Sure you can write your own Read/Write code
(and we did in the past), but that gets messy and is just another copy of
the information (fields) that you've already entered somewhere else. And
sure, you could have a compiler generate a Read/Write function for you, but
changing/updating that code then requires changing the compiler and
recompiling your entire code base.

No, if using suballocators (like ID did in the Quake series). In UDK
UnrealScript (which has bland reflection capabilities as far as I recall)
has special annotations to let the native core understand if something is
going to survive serialization or not. Being VM-based, UDK probably
serializes in a way similar to VM.

And for us, there's also a small amount of reflection needed for our
garbage collector...

I want to repeat my first message. There's something "reflective" already
going on in my native C++ core.
One of the question regards providing reflective features to the language.
As for the native core:

- My GC inspects objects, as every GC does. I wouldn't call it
reflection (we just need the pointers).
- I have something which might appear blandly reflective in the
component system. However, I don't inspect anything for having components
inside, I just know it when it happens.


As a side note, my core C++ system has no notion of any gameplay-specific
concept so specific examples are fine but try to keep it in perspective.

I appreciate your effort a lot.

Massimo