I have developed and posted here an in-game method for capturing 3D objects to an in-game image with a transparent background. That is to say, no backdrop, no background, perfect for use as a 2D overlay in-game.
What this is:
Turn an on screen object into an image. Use the image anywhere. Preserves all alpha information, not just simple on/off transparency.
What this is not:
It is not something you want to generate in real-time as animation. This process can take about 100ms (depends on size and complexity), where as a single frame at 60fps is only 16ms. So its too slow to do on each frame. You do this when loading or probably during live play a couple times wouldn't be noticeable.
Why do this at all:
If you have ever tried to mix 2D with 3D you'll notice all 2D is layered over 3D. Sometimes you need the 3D in front. The solution as been to make the 2D stuff 3D. This is a pain, and the minute the aspect ratio of the screen changes (different shape monitor), everything is out of place. This method makes the 3D object 2D, which is easier to work with, and gives more capability.
Other Methods:
At first glance, you could just screen grab an object on a black backdrop and use a colorkey, then paste it with the transparency flag on.
There's a big problem if the object is black, or requires subtle alpha levels.
Why not Set Camera To Image, and have the devs add a flag for transparent backdrop?
Awesome, but they'd still have to deal with the "Image Locked" error when moving that image to a memblock (even after the camera was deleted).
How does it work:
It is possible to set up a camera (off screen) that captures the object on both a black and white backdrop.
Then you can compare those images. Any Pixel that goes from pure black to pure white is transparent. Any Pixel that stays the same is opaque. You can record the pixels in a new image.
For 95% of the time this is all we need. When the object has transparency, this completely fails. We see bits of the backdrop, or other wierdness.
It is possible to identify partially transparent pixels since they differ from opaque pixels mentioned above, where they are merely slightly different in the two images. That's very easy to detect.
It turns out there's a bunch of wondrous math where you can compare a pixel from both images and determine what the correct RGBA values are. There's some loss of information when the object was drawn to screen, so this method is sometimes off by 1 or 2 values, not avoidable, and not noticeable.
Standalone Demo. Standard Version. v1.3 - No Files needed, New Optimizations
REM Project: TransparentScreenshot_Demo
REM Created: 4/27/2011 9:47:16 AM
REM Version: 1.3
REM Created By: Mage
REM ***** Main Source File *****
REM
`Setup the display
Set Display Mode 1024, 768, 32
Sync on
sync rate 0
`Setup a simple scene
Backdrop On
Make Object Cube 10, 1
Color Object 10, rgb(200,10,10)
Set ambient light 70
ShowTransparent = 1
`Handle a Simple Scene to provide the example
Do
`Record the time so we can control the frame rate
FrameTime = Timer()
`Setup some simple controls
CONTROL CAMERA USING ARROWKEYS 0, .1, 1
`Capture Image with the left mouse click
If MouseClick() = 1
If MyImage > 0
If Image Exist(MyImage) Then Delete Image MyImage
ENdIf
SnapShotTime = Timer()
MyImage = SnapShot(512, 512)
SnapShotTime = ABS(Timer() - SnapShotTime)
EndIf
`Alternate between showing image transparency on screen with right mouseclick
If MouseClick() = 2
Inc ShowTransparent
If ShowTransparent > 1 Then ShowTransparent = 0
While MouseClick() = 2
EndWhile
EndIf
`Paste the Image if it exists
If MyImage > 0 Then Paste Image MyImage, 10, screen height() - 522, ShowTransparent
`Rotate the example object
YRotate Object 10, Object Angle Y(10) + 0.1
`Display Instructions on screen.
Text 10,10, "Transparent Screenshot Demo v1.3 - By Mage"
Text 10,30, "Left Mouse: Screenshot"
Text 10,40, "Right Mouse: Toggle Transparency when Pasting Image"
Text 10,50, "Arrow Keys: Movement"
Text 10,60, "Time Taken: " + Str$(SnapShotTime) + "ms"
`Update the screen
Sync Mask 0x00000001
sync
`Limit the frame rate otherwise movement and rotations will be too fast.
If Abs(FrameTime - Timer()) < 16 Then Sleep Abs(Abs(FrameTime - Timer()) - 16)
loop
`This function sets up 2 cameras to Take a single screen grab with a transparent background.
`The same scene is captured with a pure white and pure black background, then both images are compared to produce the final image.
Function SnapShot(sizeX, sizeY)
`Setup Control Variables
sizeX = int(sizeX)
sizeY = int(sizeY)
ScreenSize = sizeX `Square Camera Dimensions need to fit both x and y.
If ScreenSize < sizeY Then ScreenSize = sizeY
`Select Actor
MyObject = 10
If Object Exist(MyObject) = 0 Then ExitFunction -1
`Select ScreenImage
BlackImg = GetFreeObj(0)
WhiteImg = GetFreeObj(1)
`Prepare Actor - Must be in view of cameras that are defined below.
`Move an object in view of camera and save location to move it back later.
aX# = Object Angle X(MyObject)
aY# = Object Angle Y(MyObject)
aZ# = Object Angle Z(MyObject)
pX# = Object Position X(MyObject)
pY# = Object Position Y(MyObject)
pZ# = Object Position Z(MyObject)
Position Object MyObject, 0, 10001, 0
`Rotate Object MyObject, 0, 0, 0
`Generate Black Image
Make Camera 1
SET CAMERA ASPECT 1, 1
Set Camera Fov 1,60
Set camera Range 1, 1, 50
Position Camera 1, 0, 10000 ,-5
Backdrop on 1
Color Backdrop 1, RGB(0,0,0)
Set Camera to Image 1, BlackImg, ScreenSize, ScreenSize
`Generate White Image
Make Camera 2
SET CAMERA ASPECT 2, 1
Set Camera Fov 2,60
Set camera Range 2, 1, 50
Position Camera 2, 0, 10000 ,-5
Backdrop on 2
Color Backdrop 2, RGB(255,255,255)
Set Camera to Image 2, WhiteImg, ScreenSize, ScreenSize
Sync Mask 0x00000006
fastSync
`Above: Use Separate Cameras because DBPro acts strange when trying to set same camera to new image.
`Cleanup
Position Object MyObject, pX#, pY#, pZ#
Rotate Object MyObject, aX#, aY#, aZ#
Set Current Camera 0
Delete Camera 1
Delete Camera 2
`Above: Even though the cameras are deleted the images are still "System Locked", and cant be modified (sucks). So they need to be copied to new images.
`Below: Use method of pasting images to screen and screen grabbing them to work around the "System Locked Image" limitation.
`Copy existing piece of screen to cover up pasted images afterwards
oImg = GetFreeObj(0)
GET IMAGE oImg, 0, 0, ScreenSize, ScreenSize
`Copy and crop Black Image
bImg = GetFreeObj(0)
Paste Image BlackImg, 0, 0
GET IMAGE bImg, Int((ScreenSize / 2.0) - (sizeX / 2.0)), Int((ScreenSize / 2.0) - (sizeY / 2.0)), Int((ScreenSize / 2.0) + (sizeX / 2.0)), Int((ScreenSize / 2.0) + (sizeY / 2.0))
`Copy and crop White Image
wImg = GetFreeObj(0)
Paste Image WhiteImg, 0, 0
GET IMAGE wImg, Int((ScreenSize / 2.0) - (sizeX / 2.0)), Int((ScreenSize / 2.0) - (sizeY / 2.0)), Int((ScreenSize / 2.0) + (sizeX / 2.0)), Int((ScreenSize / 2.0) + (sizeY / 2.0))
`Paste Original screen contents to cover the ugly square I made on the screen.
Paste Image oImg, 0, 0
`Delete the now uneeded system locked images and the uneeded screen grab.
Delete Image BlackImg
Delete Image WhiteImg
Delete Image oImg
`Generate Tansparent Background Render, will replace the supplied Black Image.
Snapshot_Combine(bImg, wImg, ScreenSize)
`Delete White Image Copy.
Delete Image wImg
EndFunction bImg
`This Function Uses takes two images of a scene (black and white backgrounds) and outputs a transparent background copy in place of the black image.
function Snapshot_Combine(bImg, wImg, Width)
`Exit if one of the images doesnt exist
If bImg < 1 then ExitFunction
If Image Exist(bImg) = 0 then ExitFunction
If wImg < 1 then ExitFunction
If Image Exist(wImg) = 0 then ExitFunction
`Find unused memblocks
Memblock1 = 1
repeat
inc Memblock1
until memblock exist(Memblock1) = 0
Memblock2 = 2
repeat
inc Memblock2
until memblock exist(Memblock2) = 0
`Create 2 Memblocks from black image and white image.
make memblock from image Memblock1, bImg
Width = memblock dword(Memblock1,0)
Height = memblock dword(Memblock1,4)
Depth = memblock dword(Memblock1,8)
make memblock from image Memblock2, wImg
Width2 = memblock dword(Memblock2,0)
Height2 = memblock dword(Memblock2,4)
Depth2 = memblock dword(Memblock2,8)
`Exit if one of the images has an illegal dimension.
If Width = 0 then ExitFunction
If Width2 = 0 then ExitFunction
If Height = 0 then ExitFunction
If Height2 = 0 then ExitFunction
If Depth <> 32 then ExitFunction
If Depth2 <> 32 then ExitFunction
`Nested Loops: Run Through every pixel, generating the final render, overwriting the black image memblock.
Position = 12 `skip header
mCounter = 1
for y = 1 to Height `For: Cycle pixel rows.
for x = 1 to Width`For: Cycle pixel columns.
`Retrieve the ABGR values (RGB with Alpha, in a wierd order) from the Memblocked Images.
Alpha = memblock byte(Memblock1, Position+3)
Blue = memblock byte(Memblock1,Position)
Green = memblock byte(Memblock1,Position+1)
Red = memblock byte(Memblock1,Position+2)
oAlpha = memblock byte(Memblock2, Position+3)
oBlue = memblock byte(Memblock2,Position)
oGreen = memblock byte(Memblock2,Position+1)
oRed = memblock byte(Memblock2,Position+2)
`IF Block: Find the ABGR Value for the Final Render pixel. Optimizes for Completely Transparent, Completely Opaque, and Parital Transparency pixels.
`:A pixel (x,y) from Black Image and White Image is compared to determine what the Final Render Pixel is.
If Red + Green + Blue = 0 and oRed + oGreen + oBlue = 765 `IF: The pixel is empty. Record Empty Pixel. (Black Image Pixel is RGB(0,0,0) and White Image Pixel is RGB(255,255,255))
newCol = (0 << 24) + (0 << 16) + (0 << 8) + 0
Else `Else: Pixel is not empty.
If Red = oRed and Blue = oBlue and Green = oGreen `IF: The pixel is 100% Opaque. Record the Pixel. (Both black image and white image pixels are identical)
newCol = (Alpha << 24) + (Red << 16) + (Green << 8) + Blue `//argb
Else `Else: Pixel is not 100% opaque and not empty. Oh no math... figure out what the heck its RGBA values are.
`Reset the variables that need resetting.
ColorDeviation# = 257.0
RealAlpha# = 0.0
`Use Red Color value to compute the Alpha Value
RealAlpha# = 255.0 + Red - oRed
If RealAlpha# < 0 Then RealAlpha# = 0
If RealAlpha# > 255 Then RealAlpha# = 255
`If Block: Use the Alpha value to calculate the RGB values from the Black Image Pixel.
If RealAlpha# = 0 `Error Trap: Division by 0
RealRed = 0
RealGreen = 0
RealBlue = 0
Else
RealRed = Int(255.0 * Red / RealAlpha#)
RealGreen = Int(255.0 * Green / RealAlpha#)
RealBlue = Int(255.0 * Blue / RealAlpha#)
EndIf
`Record the ABGR semi-transparent pixel for final render.
newCol = (Int(RealAlpha#) << 24) + (RealRed << 16) + (RealGreen << 8) + RealBlue `//argb
EndIf `Pixel is Opaque or Translucent
EndIf `Pixel is Empty
`Write the Final Render Pixel, overwriting the existing Black Image Pixel.
write memblock Dword Memblock1,Position, newCol
inc Position,4
next x
next y
`Overwrite Black Image with the Final Render Image.
make image from memblock bImg,Memblock1
`Delete the memblocks
delete memblock Memblock1
delete memblock Memblock2
EndFunction
rem returns next free object starting from free object marker (10000)
Function GetFreeObj(MyCount)
RetVal = 10000
While (MyCount => 0)
Repeat
RetVal = RetVal + 1
Until Object Exist(RetVal) = 0 And Image Exist(RetVal) = 0
MyCount = MyCount - 1
EndWhile
EndFunction RetVal
REM Project: TransparentScreenshot_Demo
REM Created: 4/27/2011 9:47:16 AM
REM Version: 1.3b
REM Created By: Mage
REM ***** Main Source File *****
REM
`Setup the display
Set Display Mode 1024, 768, 32
Sync on
sync rate 0
`Setup a simple scene
Backdrop On
Make Object Cube 10, 1
Color Object 10, rgb(200,10,10)
Set ambient light 70
Ghost Object On 10
ShowTransparent = 1
`Handle a Simple Scene to provide the example
Do
`Record the time so we can control the frame rate
FrameTime = Timer()
`Setup some simple controls
CONTROL CAMERA USING ARROWKEYS 0, .1, 1
`Capture Image with the left mouse click
If MouseClick() = 1
If MyImage > 0
If Image Exist(MyImage) Then Delete Image MyImage
ENdIf
SnapShotTime = Timer()
MyImage = SnapShot(512, 512)
SnapShotTime = ABS(Timer() - SnapShotTime)
EndIf
`Alternate between showing image transparency on screen with right mouseclick
If MouseClick() = 2
Inc ShowTransparent
If ShowTransparent > 1 Then ShowTransparent = 0
While MouseClick() = 2
EndWhile
EndIf
`Paste the Image if it exists
If MyImage > 0 Then Paste Image MyImage, 10, screen height() - 522, ShowTransparent
`Rotate the example object
YRotate Object 10, Object Angle Y(10) + 0.1
`Display Instructions on screen.
Text 10,10, "Transparent Screenshot Demo v1.3b - Image Kit Edition - By Mage"
Text 10,30, "Left Mouse: Screenshot"
Text 10,40, "Right Mouse: Toggle Transparency when Pasting Image"
Text 10,50, "Arrow Keys: Movement"
Text 10,60, "Time Taken: " + Str$(SnapShotTime) + "ms"
`Update the screen
Sync Mask 0x00000001
sync
`Limit the frame rate otherwise movement and rotations will be too fast.
If Abs(FrameTime - Timer()) < 16 Then Sleep Abs(Abs(FrameTime - Timer()) - 16)
loop
`This function sets up 2 cameras to Take a single screen grab with a transparent background.
`The same scene is captured with a pure white and pure black background, then both images are compared to produce the final image.
Function SnapShot(sX#, sY#)
`Setup Control Variables
sizeX = int(sX#)
sizeY = int(sY#)
ScreenSize = sizeX `Square Camera Dimensions need to fit both x and y.
If ScreenSize < sizeY Then ScreenSize = sizeY
AspectRatio# = sX# / sY#
`Select Actor
MyObject = 10
If Object Exist(MyObject) = 0 Then ExitFunction -1
`Illegal Dimensions - Limited by use of Get Image commands below.
If sizeX > Screen Height() Then ExitFunction -1
If sizeX > Screen Width() Then ExitFunction -1
If sizeY > Screen Height() Then ExitFunction -1
If sizeY > Screen Width() Then ExitFunction -1
If sizeX < 1 Then ExitFunction -1
If sizeY < 1 Then ExitFunction -1
`Select ScreenImage
BlackImg = GetFreeObj(0)
WhiteImg = GetFreeObj(1)
`Prepare Actor - Must be in view of cameras that are defined below.
`Move an object in view of camera and save location to move it back later.
aX# = Object Angle X(MyObject)
aY# = Object Angle Y(MyObject)
aZ# = Object Angle Z(MyObject)
pX# = Object Position X(MyObject)
pY# = Object Position Y(MyObject)
pZ# = Object Position Z(MyObject)
Position Object MyObject, 0, 10001, 0
`Rotate Object MyObject, 0, 0, 0
`Generate Black Image
Make Camera 1
SET CAMERA ASPECT 1, AspectRatio#
Set Camera Fov 1,60
Set camera Range 1, 1, 50
Position Camera 1, 0, 10000 ,-5
Backdrop on 1
Color Backdrop 1, RGB(0,0,0)
Set Camera to Image 1, BlackImg, sizeX, sizeY
`Generate White Image
Make Camera 2
SET CAMERA ASPECT 2, AspectRatio#
Set Camera Fov 2,60
Set camera Range 2, 1, 50
Position Camera 2, 0, 10000 ,-5
Backdrop on 2
Color Backdrop 2, RGB(255,255,255)
Set Camera to Image 2, WhiteImg, sizeX, sizeY
Sync Mask 0x00000006
fastSync
`Above: Use Separate Cameras because DBPro acts strange when trying to set same camera to new image.
` : Even though the cameras are deleted the images are still "System Locked", and cant be modified (sucks). So Use Image Kit to work around this limitation.
`Use Image Kit to Render Final Image
IK Reset Image Kit
ik load effect "RenderTransparent.fx", 1
ik set effect texture 1, "g_Black", BlackImg
IK Set Blend Mode 19
wImg = GetFreeObj(0)
IK Create Render Target wImg, sizeX, sizeY
IK Paste Image On Image WhiteImg, wImg, 0, 0, sizeX, sizeY, 1
IK Delete Effect 1
IK Reset Image Kit
`Cleanup
Position Object MyObject, pX#, pY#, pZ#
Rotate Object MyObject, aX#, aY#, aZ#
Set Current Camera 0
Delete Camera 1
Delete Camera 2
Delete Image BlackImg
Delete Image WhiteImg
EndFunction wImg
rem returns next free object starting from free object marker (10000)
Function GetFreeObj(MyCount)
RetVal = 10000
While (MyCount => 0)
Repeat
RetVal = RetVal + 1
Until Object Exist(RetVal) = 0 And Image Exist(RetVal) = 0
MyCount = MyCount - 1
EndWhile
EndFunction RetVal
// RenderTransparent.fx
// Written By Sven B
Texture g_White;
sampler2D sWhite = sampler_state {
Texture = <g_White>;
MinFilter = LINEAR;
MagFilter = LINEAR;
MipFilter = NONE;
AddressU = CLAMP;
AddressV = CLAMP;
};
Texture g_Black;
sampler2D sBlack = sampler_state {
Texture = <g_Black>;
MinFilter = LINEAR;
MagFilter = LINEAR;
MipFilter = NONE;
AddressU = CLAMP;
AddressV = CLAMP;
};
float4 RenderTransparent ( float2 texCoords : TEXCOORD0 ) : COLOR
{
// transform coordinates
float4 ColorWhite = tex2D(sWhite, texCoords);
float4 ColorBlack = tex2D(sBlack, texCoords);
// Calclate alpha (using red)
float Alpha = 1.0f + ColorBlack.r - ColorWhite.r;
// Depending on alpha, return color
if (Alpha < 1.0f / 256.0f)
return float4(0.0f, 0.0f, 0.0f, 0.0f);
// Return calculated color
ColorBlack = ColorBlack / Alpha;
ColorBlack.a = Alpha;
return ColorBlack;
}
technique RenderTrans
{
pass p0
{
PixelShader = compile ps_2_0 RenderTransparent();
}
}
[/b]Update: Latest Versions Added[/b]
Standard Version - Requires no files or plugins. Full functionality.
Image Kit Version - Requires Image Kit v2. More than 6x Faster!
http://forum.thegamecreators.com/?m=forum_view&t=176270&b=8
Any errors, oversights, or improvements are welcome.
