Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Newcomers DBPro Corner / Best way to create huge terrains in DB Pro?

Author
Message
Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 16th Aug 2010 00:23
I am looking for the most efficient way to create a terrain in DB Pro with an area of 100,000 to 200,000 square kilometres (38,610 to 77,220 square miles) and elevation markers every 4 to 5 metres/yards.

There would be villages, towns, cities and dungeons spread all over that huge terrain with a lot of NPCs, as in Daggerfall.

The player's character would be able to walk or use an appropriate (land, air or water) vehicle to travel around any part of the map, as in World of Warcraft.

From what I can gather, there are 3 methods which can be used to create terrains in DB Pro: matrices, meshes and heightmaps. Unfortunately, I haven't been able to find out how they compare in terms of memory requirements, performance, data recycling etc.

Can anyone share their experiences or point me in the right direction?
Indicium
16
Years of Service
User Offline
Joined: 26th May 2008
Location:
Posted: 16th Aug 2010 01:42
How big would these terrains be in terms of dbpro units? considering l3dt creates average heightmaps at about 0.5x0.5 kilometres, that's not gunna fit your scale. :p It all depends how you scale everything else. Personally I'd advise heightmaps because i'm in love with them at the minute, and it's easy to get the height of any point. If you want to use heightmaps, I advise you look in the WIP board at BlitzTerrain. It supports LOD and a few other things to improve performance massively over standard dbpro commands. As for matricies... If you mean a matrix ( i'm stupid xD ) then i wouldn't go with that, I get awful perfomance with them.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 16th Aug 2010 03:22
I'm not sure how big a DB Pro unit is. If you mean how many height/elevation markers there would be, it would depend on the shape of the map I end up with.

If it were 400 by 400 km, there would be 80,000 to 100,000 units per side. An 800 by 200 km map would have double that number on one side (and half on the other).

My main concern with using heightmaps is that the Advanced Terrain system seems to expect you to want to stretch a massive texture over the whole terrain, rather than using (and reusing) smaller ones on each tile/quad (or whatever the correct term is).

I did read through the BlitzTerrain thread, but it wasn't obvious to me whether that has the same drawback.

Matrices is the plural of matrix.
Indicium
16
Years of Service
User Offline
Joined: 26th May 2008
Location:
Posted: 16th Aug 2010 21:43
Dbpro units are what you see when... for example you make a cube.. make cube 1,1 with the size being 1, it is one unit big

As for the texture problem, you have a main colour map, and a smaller detail map that you can tile.

KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 16th Aug 2010 21:58
I would wait for BlitzTerrain to be completed for a terrain that large. Then you can use Terrain Streaming and make it as big as you want.

However, depending on your scale, BlitzTerrain might work for you now.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 17th Aug 2010 01:39
Mad Nightmare: I haven't used 3D graphics much, so I'm a little vague on how I would decide on a scale factor. I would guess 1 DB Pro unit would equal 1 centimetre, so a 400 km by 400km area would be 80,000,000 to 100,000,000 units per side.

Unless I misunderstood the documents I read, when using Advanced Terrain you can only have one detail texture (tiled many times), so the terrain looks the same all over. You cannot - for instance - have one texture for grass, another for dirt, another for rock etc. and place them on any tile/quad you choose.


KISTech: I'm a little wary of waiting for programs and addons to be completed, as I've been burned before.

I have downloaded the RC1 version from the BlitzTerrain thread and will certainly play with it, but I will still be looking at other options until a final version is released.
Indicium
16
Years of Service
User Offline
Joined: 26th May 2008
Location:
Posted: 17th Aug 2010 19:26
Owch, Wysardry, 100,000,000 units would practically kill dbpro. Floating point inaccuracies kick in around there, i think. For example my game of around 1km squared is about 1000 by 1000 units. So on my scale, 1 unit = 1m, so on yours, you'll need a 400,000x400,000 map if you were to use the same scale.

Now I see what you mean, sorry I misunderstood. This is a problem i realised myself, but it's a planned feature for BlitzTerrain.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 17th Aug 2010 22:06
I don't have a lot of experience with 3D graphics, but I imagine that the unit size affects the placement of objects as well as their size.

The player will be able to walk around the game, interacting with objects potentially as small as a coin using a first person perspective, 1 cm seems as large as I can make the unit size and still be able to get away with using integers.

I think Daggerfall (which ran on a 486) got around the floating point accuracy problem by splitting the terrain/map into pieces and using an independent coordinate system for each one. As far as the player was concerned it was all one seamless map though.

I don't know if that is considered a standard feature of terrain paging these days.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 18th Aug 2010 01:33
Another workaround thats been done before is keeping the player at 0,0,0 and moving everything else around them. They can turn freely in the world, but the world and everything in it moves in the reverse of the direction the player wants to go.

The other technique I've heard of is doing your world in 10,000 X 10,000 unit chunks, and as your player reaches the edge of a chunk, you move everything 10,000 units including the player. So now they're running towards 0,0,0 from the opposite side.

There's probably half a dozen other ways. (or more)

Beta 2 of BlitzTerrain is out if you haven't checked it out in a while. The latest includes RTTMS commands. Terrain streaming isn't to far away now I don't think.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 18th Aug 2010 02:29
I've heard of moving the world instead of the player, but never really understood how that would solve anything.

Surely that would mean updating many sets of variables (with potential accuracy problems) instead of one? How would having the point of origin 100,000 units from the player be any different to having the player 100,000 units from the point of origin?

BlitzTerrain 2.00 R2 has just been released, so I'm going to take a look at that. I only got as far as reading the help files for R1 as I've been reading up on matrices, memblock meshes etc. on the forums.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 18th Aug 2010 20:14
The main thing is, at somewhere past 10,000 units, floating point accuracy drops to a point where dynamic objects begin to "jitter".

So if the player never leaves 0,0,0 and you move everything else around him, as the dynamic objects move closer to 0,0,0 they get their accuracy back and stop jittering.

For static objects it doesn't matter because they are just placed where they are and nothing interacts with their position to cause the decimal portion of their position to fluctuate.

BlitzTerrain will solve your extra large terrain issues, but for other objects you'll still have the same problems.

Quote: "Surely that would mean updating many sets of variables (with potential accuracy problems) instead of one? How would having the point of origin 100,000 units from the player be any different to having the player 100,000 units from the point of origin?"


I think you misinterpreted something. The object is to make sure the player is never more than 10,000 units away from 0,0,0. As they approach 10,000 in any direction you move everything including the player 20,000 units in the opposite direction which keeps them inside that 10,000 x 10,000 area as they continue moving forward.

[example]
- The player is running towards 0,0,10000
- When he reaches 0,0,10000 you subtract 20000 from the Z position of everything and move it there
- Now the player is at 0,0,-10000 still running in the same direction with all the same objects around him
[/example]

For accurate object placement, you would probably have to devise a system where you break up your world into sectors. You would only load the objects that are in the immediate sector, and the ones around it. Those objects would have position data within the 10,000 x 10,000 sector space (to maintain their accuracy) and would be positioned relative to the current sector that the player is in.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 19th Aug 2010 05:37
Hmm. My immediate reaction to that was to wonder, "What if the player kept walking/running back and forth across the 10,000 unit boundary?" If they were travelling along it, rather than across, they could keep triggering the "world moving" code.

Is there a reason why 10,000 couldn't be subtracted instead of 20,000? I know that would mean the code would be triggered after they'd travelled only half the distance, but that would normally take them a long while.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 19th Aug 2010 21:10
Moving everything half the distance, instead of all the way across would solve that issue. Good thought!

Neuro Fuzzy
17
Years of Service
User Offline
Joined: 11th Jun 2007
Location:
Posted: 19th Aug 2010 23:08
Quote: "I don't have a lot of experience with 3D graphics, but I imagine that the unit size affects the placement of objects as well as their size."

basically, floating point values (ones with decimal places) are stored in scientific notation. They have a certain number of bits allocated for the exponent, and a certain number of bits for the mantissa. That means if you have a number that's 3.123123123123*2^200 and you try to change that value by 1, nothing is going to happen, because 1 is so so small compared to that giant number. This manifests itself as jaggedness or jumping, and it can be seen when there are objects really far from the origin.


Coding huge terrains like this is a difficult task. If you don't need to see very far, you can probably implement streaming terrain pretty easily. Having the player not straying too far from the origin can be solved by moving the terrain instead of the player (or rather, moving everything so the player is back at the origin, and just use relative coordinates).

Otherwise, prepare to code a lot.


Is't life, I ask, is't even prudence, to bore thyself and bore thy students?
Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 20th Aug 2010 00:18
Is there no way to force the use of integers for 3D measurements? An integer would allow increments of 1 mm over 400 km, or 0.1 mm if 0,0 was in the centre (instead of one end).
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 20th Aug 2010 01:41
This is the way I envision how things work concerning object placement and floats.. It's not necessarily accurate, but it makes sense given the information I've read and the things I've done on this subject over the last 4 years.

When using floats, there can be 38 digits in the number, but the accuracy is limited to the first 7 digits. The remaining digits are who knows what, and will change with each calculation.

So with a number like

0.4751237497278346108927365189768971623

Only 0.475123 is accurate.

As you add numbers to the left side of the decimal, you then lose precision on the right side. So adding 1000 to the number above effectively gives you..

1000.475

Which, if you placed your object at 0.475123 and moved it 1000 units, it's now at 1000.475, which is obviously slightly less accurate. The further out you go, the more accuracy you lose.

When you process the position of an object every loop, the inaccurate portion of the floating point number fluctuates with each calculation cycle it goes through, causing the object to be continuously repositioned in a "slightly" different place each time giving the appearance of jitter. As you add digits to the left side of the decimal the position accuracy gets worse.

Integers might help the issue if you scale everything up to an appropriate level, but the functions that position objects are still expecting floats, so you are still limited to 7 digits of precision. 0 - 9,999,999

A solution that would give some more room for movement would be to use double floats giving 15 digits of precision, but in order to do that you would have to have access to the engine source code, and change those expectations in each function that takes an object's position as an argument.

As far as I know, every game engine out there uses floats for object positioning, and they use other techniques to represent huge areas.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 21st Aug 2010 22:27
From the info I have managed to find, it seems DirectX expects single float values to be passed to it, so storing coordinates as doubles in the main memory or on disk probably wouldn't help much, unfortunately.

I'm not sure about graphic cards, but even if they did support double floats, interacting with the GPU directly would make the process even more complicated.

Maybe it would help to use doubles for any calculations handled by the CPU and then convert the final results to singles before sending them to the graphics pipeline.

Using doubles (or maybe even integers) to store the positions of objects that are outside the area being processed by the graphics card should work though.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 22nd Aug 2010 03:03
It would be easier to use one of the earlier techniques mentioned. Just go ahead and use the floats, but keep the player inside the boundaries of a specific area around 0,0,0.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 22nd Aug 2010 03:38
I'm concerned about precision errors creeping in from subtracting 5000 (or whatever) from the positions of all the objects in the world each time the player enters a new area.

If the player turned around and headed back the way they came from, I wouldn't want them to find objects that were touching have intersected each other.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 22nd Aug 2010 20:33
When you store their positions in the database, you'll be storing their position while it's inside that 10,000 x 10,000 area. So it's accuracy will be good. When those objects are outside that area, the drop in accurate positioning is actually very small, so no one will ever notice it move slightly as it's transitioned from being inside that area, to being moved 5,000 units one way or the other.

You would have to go out to 1,000,000 for the difference to be just 1 DBP unit.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 23rd Aug 2010 02:14
The player could easily go way beyond 1,000,000 units from the absolute point of origin at the scale I mentioned earlier.

I think I might have a solution - albeit one that would make collision detection more complex.

If the map were divided into sections, each one could have a unique identifier (a byte or word), like the grid references on a paper map. I'd just have to keep track of which square each object was in and it's relative position from 0,0 of that square.

Absolute positions could be calculated from that info, if needed.

I might even be able to get away with only updating the sections of the map that are potentially visible or which may cause collisions.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 23rd Aug 2010 22:07
I believe I suggested that method, but press on. It sounds like you're on the right track.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 24th Aug 2010 03:37
When you mentioned breaking the world into sectors, I thought you meant for paging and LOD purposes, not so that each part would have an independent coordinate system.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 24th Aug 2010 18:48
Ah. Sorry. I guess I could have explained it better and in more detail.

I was actually meaning both. In fact, I'm using the same technique for my space game. The coordinates for all the planets, moons, stations, asteroids, etc.. are all within a 400,000 x 400,000 x 400,000 area. For some reason I don't get any jitter. I couldn't tell you exactly why, but I'm not going to complain.

The player is always in that 400,000 x 400,000 x 400,000 area, and as they approach the edge I move them to the opposite edge. At the same time I unload all of the objects from the sector they were in, and start loading the objects in the sector they are entering, loading the ones closest to them first. I spread out the loading of all the objects in the sector over approximately 30 seconds, so their system doesn't bog down trying to load everything at once.

On a fast system it's barely noticeable. On a slower system there are slight pauses when a planet and it's moons get loaded at the same time.

Avo
14
Years of Service
User Offline
Joined: 12th Jun 2010
Location:
Posted: 24th Aug 2010 23:33 Edited at: 24th Aug 2010 23:34
Im not sure but i hear MMO's like WOW manage there huge terrains by stitching together much smaller ones. Instead of trying to create 1 massive world without lag i think you should be trying to create a 3x3 grid of much smaller "nodes". When you enter the next node delete the node behind the node you just left. Somthing like this..

#: Ungenerated node
@: Generated node
*: Players node

#####
#@@@#
#@* @#
#@@@#
#####

When the player moved into the next node forward the node at the back would be deleted with a new one being generated/loaded at the front. This would give you a way to create an insanely large world laglessly however if a player moved to fast/nodes were to small the games frame rate could plummet.

Edit: Oh you already came to the same conclusion
Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 25th Aug 2010 01:27
I found some useful info on the Virtual Terrain Project site, particularly on the LOD Papers page.

Some of the links are broken, but I was able to find most of the missing ones by pasting the titles in Google.

The method I was toying with seems to be along the same lines as the one described in The Continuous World of Dungeon Siege (PDF file). I didn't find that paper until yesterday though.

One other idea I thought of was to use hexagon tiles instead of squares, so that each LOD view was nearer to being circular. The coordinate system would be more complicated that way though.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 25th Aug 2010 04:14
I'm still holding out for BlitzTerrain for my upcoming FPS project. It's one thing to try and do this in space, where all you have to worry about are the objects floating around out there, but terrain throws a great big monkey wrench into the whole darn thing.

It looks like Blitz might be coming out with terrain streaming within the next few months, which is what I've been waiting for.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 26th Aug 2010 12:15
I found an article covering several methods of data streaming. It's more of an overview of the options than an in-depth description with code though.

Even if I were to wait for BlitzTerrain to include paging/streaming, there is no guarantee it would be in a form that would be suitable for the type of game I'm planning.
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 26th Aug 2010 19:07
That's a pretty good article. I wish I had read something like that before starting Worlds Apart Online. Although, at the moment it's being redesigned and rewritten in C++, so it will help anyway.

Where terrain is involved, I can't think of a situation where Blitz wouldn't be ideal. Given that it's many times faster than any other terrain solution for DBPro, and has the ability now for HUGE terrains, with streaming terrains just around the corner.

However, every designer has a vision of how they want things, and perhaps something about Blitz doesn't quite fit with yours. I respect that, and will be very interested in what you come up with.

Wysardry
14
Years of Service
User Offline
Joined: 10th Jun 2010
Location: Yew Kay
Posted: 27th Aug 2010 02:26
It isn't so much that I don't think BlitzTerrain will eventually be suitable for my needs, more that it doesn't have all the necessary features yet. I will be keeping an eye on its progress, but I'm not counting on it solving the problem.

As that article shows, there are several streaming methods, and the developer may choose to implement one intended for a different purpose than my own. It wouldn't be the first time.

I spent several hundred dollars on an early adopter license for a game engine in pre-alpha state as the developers said that when finished the only limits on terrain size would be disk space. A final version was eventually released, but the latest version still doesn't work 100% correctly and has now been superceded by another engine.

I have access to the source code, but am not proficient with C++ and most other users are building much smaller terrains so they haven't needed to solve the problem.

I have been watching the progress of OGRE for over a year now, as that looks like it may be suitable when version 2.0 is released. Versions 1.8 and 1.9 have to be released first though, so it's probably going to be a 2-3 year wait.

I originally purchased DB Pro (well, DarkGAME Studio actually) because I have used other versions of BASIC and wanted to try out some of my ideas using a high level language. I figured I could move to DarkGDK or PureGDK if DB Pro wasn't fast enough.

I'll probably have to bite the bullet and learn C++ eventually, but I don't really want to be doing that and trying new things at the same time.

Login to post a reply

Server time is: 2024-09-28 20:28:27
Your offset time is: 2024-09-28 20:28:27