Moved blog over to joomla

So i moved all of my blog articles over to joomla. At first it felt joomla was pretty limited, but apparently I just didn’t know how to do it. I’ll have to move my blog roll over too which will be way better because then I can have a single link for the blog roll and research links, etc.

Projects in the future

I realize that I’ve attempted to blog about certain things, then failed to deliver. I’ve been rather busy lately. Usually I’ll get on a kick to do something, then get caught up in something else. I’m sorry about that. So, for now, I’ve decided to NOT continue the multi-threading stuff. Instead, I’ll be posting my progress on my game dev tools.

Also, if I finish a book, I may blog about it in here as I don’t think that would take a whole lot of time.

I should have some time coming up here soon as I am about to be laid off at the end of this week. So hopefully something good will come of it. I will probably add a specific page that has a list of useful game dev links on it that I regularly follow or find useful.

Other projects I plan on pursuing eventually are the following:

  1. Library of common routines (basically reimplement STL with extras). I’m about 30% completed with it. This also comes with other things such as a logger, debugger, profiler, etc. Those shouldn’t take too long.
  2. A simple OS of some sort. I need to do more research, but I have plenty of notes on it. Honestly, this shouldn’t take TOO long. I generally have an idea of how to go about doing it. I need to do more research on it though.
  3. Complete tile-based 2D game engine and level editor (See previous post). Will be done in XNA & C#.
  4. Robust multi-threaded game engine with all the bells and whistles. (SSAO,hdr,deferred ?, other cool stuff, task-oriented/component/flat architecture, animation, A*, outdoor terrain and indoor areas, robust physics system, spatial partitioning, scene graph, etc.) This one is dependent upon bullet #1 though. The multi-threading posts will come from this.
  5. Various emulators. I want to start small. I generally get the idea, but emulating the NES & SNES will be a little more difficult. So I’ll work on really simple microprocessors first such as the Z80 or something, then work to the ATARI 2600, etc. I imagine this will take longer than expected.
  6. I’d like to learn F# as it is a functional language. F# or Erlang. I’ve actually seen a physics engine in Erlang, so I know it is a capable language.
  7. Add more cool features to my ray tracer.
  8. Compiler & a scripting language / interpreter.

One thing that I would like to do is make #4 also have a software graphics pipe as well as OGL & D3D. Obviously this will take a long time to do, but I hope to make it modular enough to swap these components in and out without any code changes. Most of the nice graphics I plan on doing in D3D though. Unfortunately, I won’t be able to make it DX11 compatible until I actually get a DX11 card.

I have a lot of projects on my plate, but these are all things I feel that would be sweet to implement. Most of these projects I know how to go about doing, the only problem is actually finding time to do them. Between work, playing games, and doing nothing, I need to find time. :-)

I also can’t seem to find a theme for my blog that I like.

Indie Game Team

It has been a long time since I’ve updated my blog, so I thought I should update it. No one probably reads it anyways, but I’ll just keep blogging anyways.

So I am currently developing a game (xbla probably) with a team of people. We have 2 designers, 2 engineers (me included), & 2 artists. I think we can make something really cool happen with it. We decided to make it a 2D side scrolling game with a deep story. The artists we have are talented, so I’m no longer worried about it looking like poop. The first task though will be to develop a level editor for the game. Since we decided to go with XNA, it also makes sense to make the level editor in C# as well.

This has a few interesting bits as some of the components will be modular and I plan on making them available to both the game and the level editor. I will keep updating the blog with our progress and discuss certain areas of the engine I feel were kind of interesting.

What is endianness?

What does it mean for the byte order to be little-endian versus big-endian? Basically, it’s how the computer interprets multi-byte values. For example, we know that a single byte can take on the values of 0-255 inclusive. The arrangement of the bits within that single byte is pretty standard.

(1)2^7 + (0)2^6 + (0)2^5 + (0)2^4 + (1)2^3 + (0)2^2 + (0)2^1 + (1)2^0 = 137

Looking at the above and only looking at the ones and zeroes we have 10001001 = 137 in decimal.
We usually have the location of the smallest bit (least significant bit) on the right going to the largest bit (most significant bit) on the left. So this is for a single byte. What about multi-byte values?

First lets discuss some memory addressing issues. When you have a pointer that points to a memory address: char* p = ……; Something like that (we are assuming a char is a single byte), then p points to a single byte.

If we increment it like this: ++p, then p will point to the next byte in memory. It’s important to note that addresses in memory are the location of a single byte. You can never address bits. Although by using bit shift operators and such, you can manipulate the bits within that single byte.

Let’s expand on this a little bit. Lets say we have an array of characters:
charletters[]={‘a’,’b’,’c’,’d’};

So we have four characters contiguously stored in memory. Each character in the char[] has a number associated with the character as defined in the ASCII table. a = 0x61, b = 0x62, c = 0x63, d=0x64. I’m not going to cover hex notation since you should already be familiar with it, but just keep in mind that two hex digits represents a single byte. So 0x?? is a single byte.

So in memory, we have something like this

Address & Values
0x70 0x61
0x80 0x62
0x90 0x63
0xA0 0x64

Note that all ASCII is, is a converter from english letters to numbers. The important thing to keep in mind is that the ascii letters are just numbers or hex values, and these hex values are what is stored in memory since hex values are equivilent to binary. Endianness is just a way to interpret the bytes within a multi-byte value. These multibyte values are, for example, short (16-bits), int (32-bit), etc.

I’m mainly concerned with intel architecture here. Which by the way uses little-endianness. PowerPC on the other hand uses big-endianness. So what is the difference? The difference lies in how the architecture interprets these values in memory. Let’s look at an example by taking our four byte array and converting it to an int (32-bit = 4 bytes).

We don’t actually have to do anything since we interpret these four bytes however we want. We can interpret them as the letters a,b,c,d or as an int, or as something else. So lets figure out what this “int” is. After all, the entire purpose of this article is to discuss the differences between little and big endian. The thing to be careful though is the fact that even though our four bytes are contiguous in memory as the values 0x61, 0x62, 0x63, 0x64, interpreting these four bytes as an int will be different with respect to both little endian and big endian. So lets convert it to little endian first.

Little Endian: Littlest address = littlest bit
memory addresses: 0x70,0x80,0x90,0xA0
memory contents: 0x61, 0x62, 0x63, 0x64
number interpreted as an “int” in hex using little endian ordering: 0x64636261
conversion from the hex value into a decimal value: 1684234849

Note: even though the bytes in memory are 61,62,63,64 as the memory addresses increase, on a little endian CPU, it interprets them from the highest address being the most significant and the lowest address being the least significant.

Big Endian: littlest address = biggest bit OR biggest address = littlest bit
memory addresses: 0x70, 0x80, 0x90, 0xA0
memory contents: 0x61, 0x62, 0x63, 0x64
memory contents interpreted as an “int” in hex using big endian ordering: 0x61626364
conversion from the hex value into a decimal value: 1633837924

The decimal values are NOT the same, which means, even though the bytes might be stored exactly the same in memory, the CPU will grab the 4 bytes and based on these four bytes use little endian or big endian to determine what the 4-byte hex value is, and thus what the 4-byte decimal value is.

So, as you can see, endianness is just a matter of interpretation. After all, we started with the letters a,b,c,d contiguously in memory.

I don’t make any claims as to the accuracy of this article, but if something is incorrect, please let me know.

What am I doing?

I’ve decided to keep people informed, if anyone actually cares besides me, as to what I’m doing. Mainly it’s to keep me focused and to keep me learning. So what am I doing / learning right now?

1) Linkers and Loaders, libraries

2) Multithreading, I’ve learned the techniques, but I’ve never really used them in a real world scenario. I’m going to continue learning though and hopefully I’ll be able to use them in my own core engine.

3) Unit testing techniques, I still need to start doing this for my own projects.cxxTestorQ1is what I’ll be using, I haven’t decided. I’ll have to research Q1 a little more though.

4) Implementing a scripting language into my core engine I’m developing. I haven’t decided on lua, stackless python, or ruby? Lua seems easy, so I’ll probably start with that.

5) Learning howOgrestructures their engine. I’m hoping I can borrow some of their techniques and learn from them. Also for theQ-Gearsproject I hope to work on soon. The project has slowed but it uses Ogre internally. Learning ogre will help me in more ways than one though. I plan on writing an unpacker first of all, then some texture viewers/model viewers for FF7, just to get up to speed on things. We’ll see though.

Beginning win32 multithreading

In an attempt to understand windows multithreading a little more thoroughly, I’ve decided to write about the simple concepts associated with multithreading. This is mainly to clarify it in my own head but maybe it will help someone else out too. If any of this information is wrong, don’t yell at me.:-)Please just post a comment and I’ll look into it and/or fix it. So here we go.

Thread: I’m not going to define a thread. Just know that a thread is a kernel object, which means that when a thread finishes executing, it gets signaled. If you use “WaitForSingleObject();” for example to wait for it to finish executing, it is safer and doesn’t eat up CPU time like a spin lock would.

Signaled: Kernel objects get signaled (threads, mutexes, etc.). When a kernel object gets signaled, “WaitForSingleObject()” wakes up. The same goes for other “Wait*()” functions. In the case of threads: running = nonsignaled, when a thread terminates = signaled.

Atomic Operation: A single operation that the processor can not perform a context switch in between.

Context Switch: When the processor halts execution of a thread, and then continues executing another thread from where it left off previously. The cause of race conditions when multithreading, but a necessary feature on processors.

Spin Lock: Eats CPU time, not atomic. Basically the predecessor to critical sections. Basically a “for” loop that waits around until a thread returns. This is the worst way to do multithreading, don’t use spin locks at all.

Race Condition: The easiest way to conceptualize a race condition is to think of it this way: A thread writes data and soon after, another thread blindly overwrites the data that was just written by the first thread. Critical Sections resolve this.

Critical Section: If you only have one resource that needs sharing between multiple threads, you use a “Critical Section.” Critical sections are not kernel objects and prevent race conditions between multiple threads. The reason you use critical sections is to make certain areas of your code / data atomic so that a context switch doesn’t fowl up your data. Note: You can specify different areas of your code as the same critical section.

Mutex: If you have multiple resources that need to be shared among threads, you do NOT use a critical section. This can result in a dead lock where one thread waits for the other thread to release a resource, but that thread is waiting on the first thread to release a resource it needs, so both end up waiting forever. So instead you use a mutex. A mutex is a kernel object and can be shared among threads as well as processes. Basically, a mutex says: “Execute this code if both (or however many the mutex was defined with) resources are available.” If that requirement isn’t satisfied, then the code isn’t run. For example: If only one resource was available but the mutex was defined for two resources, then the code would not run.

Note: Mutexes introduce race conditions now, the very thing that critical sections fixed. Also, it’s worth noting that you can still get a dead lock with mutexes if you use “WaitForSingleObject().” Instead, you must use “WaitForMultipleObjects()” to avoid dead locks.

Win32 functions associated with multithreading
General Win32 Thread Functions: You can look up the parameters yourself on msdn.
CreateThread();
GetExitCodeThread();
ExitThread();
WaitForSingleObject();
WaitForMultipleObjects();
MsgWaitForMultipleObjects();

Critical Section: Sharing one resource
InitializeCriticalSection();
DeleteCriticalSection();
EnterCriticalSection();
LeaveCriticalSection();

Mutex: Sharing multiple resources
CreateMutex();
OpenMutex();
WaitForSingleObject();
WaitForMultipleObjects();
MsgWaitForMultipleObjects();
ReleaseMutex();
CloseHandle();

Notes on library files

A few notes regarding library files.

Shared library: Not the same as a dynamic library. Only used in linux. A .so file extension. Can be shared across processes, not loaded into a process’s address space, loaded globally. (i believe) Note: It is loaded at application startup.

Static library: Also known as an object library, contains a collection of .obj files, extension .lib

Dynamic library: .dll extension or .so. Dynamically loaded by the OS’s loader. Loaded into a process’s address space. (i think) Note: Loaded whenever you use a “load library()” call, not on application startup.

Import library: Allows your program to call entry points in the DLL, .lib file, linux does not require these as it directly links to the .so file

Sometimes a shared library and a dynamic library are said to be the same thing, but they aren’t. If there are any mistakes or I have my facts mixed up, please let me know.

In fact, if you want, there is an entire book dedicated to linkers and loaders online. And it’s free: Linkers and Loaders

The hidden default constructor in C++

Note: This posting is just completely wrong. I didn’t know C++ very well at the time, so I’m just mentioning it. I won’t bother correcting it though.

I think I’m going to start posting about general C++ stuff too and since C++ is my favorite language (aside from assembly) I thought it would be interesting to post obscure C++ things that I find. Today I’m going to talk about a blog post that I found a while ago regarding constructors.

The fellow (I forgot where I saw it) mentioned that there was a significant difference the two lines of code.

Class *pClass = new Class;
Class *pClass = new Class();

The first line will NOT initialize intrinsic data types. So they will be filled with whatever when the object is instantiated.
The second line, on the other hand, WILL initialize all intrinsic data types to zero.
There are a couple of issues to be aware of though:

1) There must NOT be a default constructor available.
2) This is very well compiler/implementation dependent. I haven’t actually checked the standard though but I don’t see why the standard would define such a thing since it could (theoretically) impose a performance risk.

Looking back at point 1, lets use an example:

Class{
int foo;
};

Note: No default constructor or destructor. In this case, the compiler will provide them for your automatically. At least Microsoft’s compiler will. In that case, it will be kind enough to also initialize foo to zero if you specify

new Class();

as opposed to

new Class;


Intellipoint – restore maximize toggle app

I bought a new mouse (Microsoft) but the Intellipoint software doesn’t allow you to perform a “Restore / Maximize” toggle. You can either restore it, or maximize the window, but not toggle. So I made a very simple app that performs the toggle. Just set Intellipoint to execute the program and it will maximize or restore the active foreground window. You can find the program here:Link

Useful links / blog roll

If you’ll look to the right side I put up a “useful links” section. It contains a lot of valuable links to current research and useful game dev sites. I’ll be updating it as time goes on.

Also, for those still looking for a game dev job, I stumbled across this guy’s blog posting regarding interviewing at game dev companies:Link

Too many white papers, too little time

I’ve been hearing quite a bit aboutO3Dfrom Google, but I’ve just sorta blown it off as another Google thing. Kind of like their Chrome Browser which I wasn’t too fond of. But I actually got a chance to check it out and I must say that I’m impressed. This demo shows some interesting bits:Beach Demo

The only thing I don’t like about it are the input controls. I couldn’t simultaneously press “w” and move the mouse at the same time. But overall it looks impressive. Eventually people will start using it for game dev. It appears they already have checkers operational. The only problem I see is that it is yet another api to learn.

On a side note, I have begun to realize, after trying to keep up with my rss feeds, that it is next to impossible to keep up to date on all the latest and coolest things. What I’m referring to are “white papers.” Some of them are fantastic but not useful, while others are very useful. So how might one go about keeping up to date with the latest, but weed out the less important white papers to avoid wasting time?

Currently, I don’t have a solution to this problem. But the best I can hope for is to only read the papers that other people recommend and/or the popular papers. Another thing to do is to avoid reading white papers all together and only read books. This presents a problem for a couple of reasons but also might be a good idea:

1) Pro: Books usually contain useful proven information (I like to call them stable solutions.) You’re much less likely to waste time if you read a book.
2) Con: You won’t get the “latest” engineering solutions. You could always engineer a solution yourself though.
3) Con: You might not be able to find what you’re looking for in books. There aren’t a whole lot of books on “point rendering” for example.

Honestly, there is too many interesting things going on to always get the best of the best. From GPGPU processing, Graphics (real-time, precomputed), volumetric rendering, etc. etc.

My Current Status

Woohoo, I finally am able to use ICS through my laptop… that’s a big achievement for me… a couple of my computers haven’t been able to access the internet in a long time.. But I finally figured it all out.. Now I can use a third monitor with Synergy and actually make it be productive. Like offload all my regular stuff, like rss reader, and notes to that workstation instead of my main computer.

Also, I found this neeto tool called flux. From stereopsys.com. A friend of mine tried it though but it didn’t seem to work. It adjusts the temperature of your monitor automatically based on the time of time. I’m not sure how useful it is, but I think I like it.

I’m also making progress on my HDR project for my cs562 class. I’m pretty sure I understand all the steps, but there’s a few bugs in it still.. Hopefully when I get that done, I’ll be able to integrate it with the Spherical Harmonics & PRT project I’ll be pursuing. Also for that class. When I’m done with everything I’ll be posting screen shots and stuff. My ultimate goal is to get the SH, PRT, Deferred Shading, and HDR stuff all integrated together. I’m not even sure it’s possible or even practical. I’ll have to do more research though.

Also, I changed the level editor for my game (Sundown Cerose). Now we have collision boundaries that can be defined within the level editor. It also is not bug free, but I don’t think it will take very long to get the bugs worked out so our artist can use the tool.

A Few Code Snippets

Periodically I’ll stumble across a small but useful piece of code. I have a small list right now, but as I stumble across them, I’ll post them here. Note: I do not take credit for any of this code.

Pausing your application

__asm int 3;

or

__debugbreak();

Convert any data type to a std::string object

#include <sstream>
template <typename T>
std::string toString( const T &arg )
{
    std::stringstream ss;
    ss << arg;
    return ss.str();
}

Test if a float is zero or near zero

inline bool IsZero( float val )
{
    return( fabs( val ) < EPSILON);
}

EPSILON is a #define that is something small, like 0.002f
Note, I will repost, I just realized that serendipity can’t handle angle brackets…