Server Updates

Temporary Login/Title Screen

So, after having good success moving the client to Unity, there were still some issues.

My primary concern was that there were now two physics engines involved:  BEPU on the server, and an old version of PhysX which Unity was running as the client.

Now, I wasn't certain that this would cause problems, but I'd done some fairly deep dives into the quirks and behaviors of BEPU, so it seemed pretty likely.

It would have been really nice to be able to use the Unity engine from the Zone Server.  If there was some way to simply include it as a DLL and run it, that would have been fantastic.  This is a familiar refrain you see all over the Unity forums, particularly the networking on.

Sadly you can't, of course, and I'm not surprised.  So, one would think that-would-be-that.

New Posts Soon

I've just finished moving, and let me say that if I don't do that again in a long time it will be too soon.

Anyway, quite a lot of interesting dev work has happened in the last week or so, but I've been too busy packing and moving to write anything.

Radical Changes

I'm making some radical changes to the development path I'm on.  I've had an increasing number of small but extremely irritating and time consuming problems with the game client.

I've decided that I just don't have enough time to work a full rendering engine as well as the back end servers, database and game administration projects.  So I'm taking a new path with the most troublesome, the game client.

A Tiny Slice of DLL Hell

So I've begun preparing for a little ultra-pre-Alpha.  No real gameplay, just a test of the networking and physics.

In order for this to proceed, though, I naturally had to test to make sure the client would work on other machines.  Also I wanted to run the servers on another machine as well.  Everything worked fine together on my Dev box... but you know how that is.

Physics Mismatch... of Bepu Physics and Small Rotations

So I've been working on syncing the physics between the client and server implementations.  It's been... a bit painful.

Varying Update Rates Hertz a Lot

One of the first things that tripped me up was the different update rates.  The client is running at the regular 60 fps, i.e. updating every 0.016ms.   The server is running a much more sedate 10 updates per second, or every .1ms.

I had assumed that passing the delta time into the physics simulation would deal with the difference.  I.e. it would know either one sixtieth of a second or one tenth of a second had passed, and simulate that amount of time passing:

physicsSpace.Update(elapsedMilliseconds);

First, this was WRONG, and a very common mistake!  Turns out Bepu wants the time in seconds, so really I'm telling it to simulate x1000 speed!  This is correct:

physicsSpace.Update(elapsedMilliseconds/1000.0f);

Why didn't I notice this right away?  Well, it turns out Bepu has a default timestep for its simulations, which happens to default to 1/60th of a second.  And it will only perform a certain amount (default three) physics steps per update.  So by default Bepu will simulate .005 seconds of time (3x0.016), maximum, per Update call.  Mind you, these are quite sensible numbers for defaults.

So really when I was passing in weird values like 16 (16 milliseconds, 1 frame at 60fps) I was actually asking it for SIXTEEN seconds.  In return it gave me 0.05 seconds, which is near enough to .16 that by the naked eye I couldn't tell the difference.  On the server it was getting 100 (100 milliseconds, the update period at 10 fps), so it though I was asking for a bit over a minute and a half...

So, problem number one was I was passing milliseconds rather than seconds.  The defaults were ok for the client, but clearly the server running at 10hz would still be wrong.  Each update would need to simulate 100 milliseconds of time, but by default Bepu would only step 50 milliseconds.

Easily fixed by a bit of reading through the docs and forums, however.  You can change the maximum number of timesteps simulated:

physicsSpace.MaximumTimeStepsPerFrame

... though I did not change this.  Really you want a one-to-one normal ratio, with the ability to simulate extras if for some reason the simulation falls behind.

More importantly, you can change the expected simulation step temporal duration:

physicsSpace.TimeStepSettings.TimeStepDuration = timeStepDuration

where physicsSpace is the Bepu object, and timeStepDuration is set in the simulation engine initialization call.  So the client and server both pass in their expected update tempo, and the physics engine deals with it.

Velocity Mismatches

That fixed some of the differences I was seeing between the client and server, but there was still one bothersome one.

For small accelleration values (angular or linear) despite the fact that I had set the entity's AngularDamping and LinearDamping values to zero (this is a space simulation after all) the AngularMomentum and LinearMomentum values were still damping down to zero over a short time... but only if they were very small.

I was deeply confused and verified again and again that the damping was zero, and kept looking for where I had gone wrong.  This was throwing off the client and server sync, ans the client seemed to damp down quite a bit quicker (which seemed like a useful clue, as I knew it ran more, but shorter, simulation steps.  So it had to be some sort of per-step, rather than per-time-unit effect).

I finally found the answer. On the Bepu forums, someone poses basically the same question, and Norbo answers:

That's caused by the deactivation system. It will attempt to stabilize and disable very slow objects. You can prevent it from freezing entirely by setting the object's IsAlwaysActive property to true. However, stabilization will still be applied. You can disable stabilization separately by setting the Space.DeactivationManager.UseStabilization to false.
Stabilization is nice to have on in the general case, though, particularly if you have complicated interactions like large stacks or constraint systems.
You can also tune the Space.DeactivationManager.LowVelocityTimeMinimum and Space.DeactivationManager.VelocityLowerLimit. Stabilization only occurs under the velocity limit, and deactivation will not occur until an object is below the velocity limit for the time minimum.

By default the velocity lower limit is .026f.  Thus I added:

physicsSpace.DeactivationManager.VelocityLowerLimit = .001f;

Well, that solved a lot of problems.

Outlining Post Process

I've been working on the (temporary simplified) screen where you select a ship.  Currently when you click a ray is cast into the scene's physics space and the first entity hit is pulled back.  That's the one you clicked on, basically.

To visually implement this I had simply tinted the selected entity red when it was drawn (from the model rendering shader):

 if (isSelected)
  output.Color.r = output.Color.r * 2;

Clearly that was a bit... simplistic, though it was enough at the time and it worked.  But now I needed something a little better.

Temporary 'Hangar' Screen

In working towards something vaguely playable, I've fleshed out the screen you get after logging in, the 'Hangar' screen.  Here you'll pick a ship type, then you'll be able to move on to actual gameplay.

When a client connects to the Hangar 'zone', the server gives it the entity list.  For now, there are three items (well, one is a group):  a destroyer, a corvette and a fighter squadron.  These are all synced from the server, so this list can easily change without any changes to the client.

Further Client Server Syncing

So last night I got basic synchronization between the client and server working.  Basically I had the server starting a zone, creating a single entity (a small spaceship) and forcing it to slowly rotate.



When the object was issued a command that updated it's attributes that data was then sent to any clients connected to the zone (GAMEOBJECT_UPDATE_MESSAGE).

Server Work

I haven't posted much in the last week or two.  I have been making progress, but largely I've been integrating the game client and simulation engine with the server system I'd written last year.  So, nothing glamorous, just a lot of 'oops!, I shouldn't have done it that way' sort of restructuring going on.

There are som interesting bits.  The simulation engine references XNA in order to use the various types (vectors, matrices, quaternions, etc,) as well as being able to load models.  Per triangle hit detection is one of the requirements, so I definitely have to have the model data and account for animation movement on the server.  So I figured I'd just reference and use XNA there, though definitely not actually running via the Game class of course.

I quickly realized that there was a serious problem.  Loading a model with XNA not only loads the data in the model file, it loads the textures and puts the model's vertices and indices in VertexBuffers and IndexBuffers as well.  Texture2Ds, VertexBuffers and IndexBuffers live on the video card.  There's definitely no guarantee that whatever machine the server is running on would have a DX9 video card; likely they'll be virtual anyway.  Plus it's just a waste.

Unfortunately there's no way to modify that loading behavior.  I considered looking up the format of the XNB files (the binary format XNA compiles resources to) and reading them directly, but then I realized that MonoGame does that already.  MonoGame is an open-source implementation of XNA.  It isn't entirely complete yet, so I can't use it on the front end, but for the server it should be entirely enough.

Even better, since it is open source, I was able to go into the ContentManager class and add some hacky code that allows to you instantiate a ContentManager and load a model without loading any shared resources (i.e. those that would reside on the video card).  Exactly what I needed!

There were some sticky problems with integrating this, however.  Since the SimulationEngine project was referenced from both the client and server, and both XNA and Monogame reside in the Microsoft.XNA.Framework namespace.... yeah, it really blew up at first.

Initially I wrote some interface code that did a bunch of casting and conversion between the two, but that was just ugly.  In the end I refactored the base SimulationEngine, making two additional projects, SimulationEngineClient and SimulationEngineServer.  Both these projects actually link to the files in SiimulationEngine, making them exact duplicates (though each overrides a few files).  Then SimulationEngineClient links to XNA, SimulationEngineServer links to Monogame.  Since they reside in the same namespace, nothing need change.  Boom, problem solved.

Anyway, each of these is inherited by RogueMoonSimulationEngineClient and RogueMoonSimulationEngineServer, which implement Rogue Moon game specific logic.

It is a bit twisty, but the code is pretty clean, and so far it hasn't presented too many problems, now that I have this all worked out.  It took me a while to get here, though!

I Find Git Very Annoying

So I've generally been working with Bitbucket and Mercurial.  Everything has been happy and working well.

But lately I've done some experimental work with Azure, and also decided to clone Monogame from GitHub.  I'm taking Monogame and making some additions so that you can load XNA Models server side.  The changes will allow you to load a model without the textures, vertex/index buffers and effects.  Which also means you don't need a graphics device.  This will let me load the model and bones server side (which is likely headless) for per-triangle hit accuracy even with animation.

Anyway, so I clone Monogame to Bitbucket.  Fine.  I then clone the repository locally.  I tried making may changes and pushing, but got an error.  Essentially the remote repository was ahead of my local (which I had JUST pulled).  Eeeeeh?

Well, it seems like what is happening is that when I clone the repository it is taking from the Develop branch.. but when I try to push it is pushing to the Master.

You'd think that would be easy to change, but no... it's probably somewhere in some arcane command line option that I'll have to dig to find.

Anyway, it doesn't just work like Mercurial has been doing.  I'm sure I can dig to make it work... but this is my first encounter with Git, and it has thusfar failed completely to do something VERY EASY.

I didn't know anything about Git previously, but I think I'll avoid it from now on if practical.  Pull->Change one line->Commit->Push ought to just work.

Edit:  I wasn't able to figure out how to do this from the command line or Git GUI.  But GitHub for Windows allowed me to easily switch the operating branch.  Twenty seconds there and I was done.  Thanks, Git.

A Context Menu

As I'm getting more and more things added into the engine, interacting with all the objects in the scene has gotten more complicated.  Also I'd like to start working toward an actual game-like interface as opposed to my cheesy test interface.

Besides all the debug-type side buttons, you could take control of any of the entities in the scene by double-clicking on them.  This allowed you to pilot them with the arrow keys (a bit cumbersome, but fine for testing physics/collisions/rendering).  A single click would select an object, highlighting it.

Forward Rendering and Transparency

After quite a lot of work into the wee hours last night I now have the Forward Rendering pipeline set up.  When an entity is created it can be given a RenderBehavior of Full, DeferredOnly or ForwardOnly.

'Full' entities are rendered via the Deferred rendering step.  They are also active in the Forward rendering step, but the entities themselves are not drawn, just (if activated) associated primitives such as bounding boxes, debug info, etc.

DeferredOnly entities are rendered via the Deferred step, and completely excluded from the Forward step.  Vice versa, ForwardOnly entities are rendered via the Forward step and completely ignored during Deferred processing.

The is one complication I do need to deal with:  an entity that is party transparent, but has non-transparent parts needs to render in both the Deferred (non-transparent parts) and Forward (only the transparent parts). I need to sort that out properly.

Cloud layer is transparent and Forward rendered.

Specular and Normal Mapping

I've finally gotten around to re-adding specular and normal (bump) mapping.  Project Vanquish, the framework I began from, implemented these.

Early on since I quite frankly had no idea what I was doing, I disabled a lot of things for simplicity.  Specular and normal mapping were among the first things to go in the Great Renderer Simplification.  They've now returned, and look quite nice!

Shiny!

Restoring Depth

I've been refactoring some of the drawing code in the Rogue Moon game engine.  I've had nice bounding box drawing working and in the same manner wanted to add some sort of 'engine trail' effect.  I got this working, but then decided that both of these needed to move to a 'post process' step.

Game Engine Part 2

As I said in the last post:

I'd been mulling the engine design over in my head while working on the graphics engine.  I wanted, above all, to make it easily extensible and cleanly separated from the rendering engine.  After all, Rogue Moon is to be an online client-server game.

Then I got off onto the tangent about game networking.  But it did have a direction.  If you followed that post, you may have realized that modern latency-compensating online games require two simultaneous (yes, I know I said simultaneity largely goes out the window) simulations, client and server.

A Game Engine

So, it's been a while since I made a post; Christmas and the like interfered a bit.  But I have done quite a lot of work.

For now the shadows and rendering engine are 'good enough', so I've moved on to the meat of the game engine, as it were.

I'd been mulling the engine design over in my head while working on the graphics engine.  I wanted, above all, to make it easily extensible and cleanly separated from the rendering engine.  After all, Rogue Moon is to be an online client-server game.

Edit:  this turned into a post about networking and game engines, not where I initially meant to go.  But I'm going with it...