Magic FVF code to fix shader woes

Author

Message

Cagedrei

15

Years of Service

User Offline

Joined: 18th Mar 2009

Location:

Posted: 19th Apr 2009 23:32 Edited at: 19th Apr 2009 23:44

Link

I've been pulling my hair out, trying to get shaders to work with imported objects. Sometimes they've worked. Other times they've crashed DBP, made my objects vanish, turned my objects into tangled messes. Frustrating. I found no discussion on the forums about how to deal with this.

I think I've found a solution. If you're having similar problems, maybe this will work for you. One line of code:

+ Code Snippet

convert object fvf 1,274 `Set to DBP default object FVF (acquired using DK get object fvf)

Replace the object number with your own. This sets your object's FVF state to match that of the default DBP primitives, with which shaders do work.

This seems fairly remedial, really. Basic stuff. One wonders why it isn't in the docs. The documentation for Convert Object FVF is misleading, IMO. As presented, it suggests that FVF settings pertain only to vertex shader effects. A newbie like myself is easily led to ignore such a function, then, until working with vertex shaders specifically. "Oh. That's vertex shaders," I said to myself. "I'm struggling with pixel shaders. Right?" The docs also present everything in hex format, which is great for those who already understand what's going on, but useless for a beginner.

+ Code Snippet

FVF stands for Flexible Vertex Format and represents how much data
 each vertex of your objects meshes contain. Use this command to
 prepare an object for use with a vertex shader effect. The FVF 
Format is specified using bits. Each bit represents an element that
 will be created within the vertex data. The FVF bits are specified
 within a DWORD and are as follows: 
FVF_XYZ=0x002,
 FVF_XYZRHW=0x004,
 FVF_XYZB1=0x006,
 FVF_XYZB2=0x008,
 FVF_XYZB3=0x00a,
 FVF_XYZB4=0x00c,
 FVF_XYZB5=0x00e,
 FVF_NORMAL=0x010,
 FVF_PSIZE=0x020,
 FVF_DIFFUSE=0x040,
 FVF_SPECULAR=0x080,
 FVF_TEX0=0x000,
 FVF_TEX1=0x100,
 FVF_TEX2=0x200,
 FVF_TEX3=0x300,
 FVF_TEX4=0x400,
 FVF_TEX5=0x500,
 FVF_TEX6=0x600,
 FVF_TEX7=0x700,
 FVF_TEX8=0x800.

Please don't think I'm whinging, but DBP seems a bit strange about things like this. Some processes are made extremely easy - apply a shader in one line of code, using Set Bump Mapping On, for instance. So the beginner may be well represented - if the easy code works. If it doesn't, one has to hunt for a solution, and the documentation can be extremely obscure. DBP seems to jump straight from beginner to advanced in such areas. I guess the forum here is meant to pick up the slack, which is why I'm posting this.

I'm trying to track down a hex table which can allow me to translate the hex values in the Convert Object FVF listings to integer values, to post here. Then we can add up the integers to set the target FVF. If newbies like me have a simplified explanation of this sort of thing, it might help all of us.

Back to top

Profile PM Email

Math89

20

Years of Service

User Offline

Joined: 23rd Jan 2004

Location: UK

Posted: 19th Apr 2009 23:49

Link

You can actually use the hex values without knowing what they represent in decimal notation. As long as you put '0x', DBPro will understand you are using hex values (for binary, use '%').

Back to top

Profile PM

Cagedrei

15

Years of Service

User Offline

Joined: 18th Mar 2009

Location:

Posted: 20th Apr 2009 11:10 Edited at: 20th Apr 2009 23:52

Link

Okay, I did some research, and I think I understand the FVF business a bit better.

The 274 magic does work, though. The "default" of 338 for imported objects is no good. (The calculator below reveals the 274 value to be convert object fvf object, 0x002 || 0x010 || 0x100.)

I put this together to try to understand the FVF hex math. It's an FVF calculator, which lets you add up your constants and see what you're getting. It can also try to break down an FVF value into its likely set of constants used for its construction.

` FVF constants - with value returned by Matrix1 plugin Get FVF Limb    
#constant FVF_XYZ = 0x002 `2 untransformed vertex - discludes FVF_XYZRHW
#constant FVF_XYZRHW = 0x004 `4 tranformed vertex - discludes FVFXYZ and FVF_NORMAL
#constant FVF_XYZB1 = 0x006 `6 position data and weighting values for vertex blending matrix calcs
#constant FVF_XYZB2 = 0x008 `8
#constant FVF_XYZB3 = 0x00a `10
#constant FVF_XYZB4 = 0x00c `12
#constant FVF_XYZB5 = 0x00e `338
#constant FVF_NORMAL = 0x010 `16 vertex normal
#constant FVF_PSIZE = 0x020 `32 vertex point size
#constant FVF_DIFFUSE = 0x040 `64 vertex diffuse color
#constant FVF_SPECULAR = 0x080 `128 vertex specular color
#constant FVF_TEX0 = 0x000 `338 Number of texture coordinate sets for vertex, 1 - 8
#constant FVF_TEX1 = 0x100 `256
#constant FVF_TEX2 = 0x200 `512
#constant FVF_TEX3 = 0x300 `768
#constant FVF_TEX4 = 0x400 `1024
#constant FVF_TEX5 = 0x500 `1280
#constant FVF_TEX6 = 0x600 `1536
#constant FVF_TEX7 = 0x700 `1792
#constant FVF_TEX8 = 0x800 `2048

remstart
    (Cagedrei's FVF notes)
    FVF value is an integer, calculated by specifying FVF constants combined using the bitwise or operation.
    The FVF value of the the DBP primitives is 274.
    The "default" FVF (covered in Make Memblock From Mesh documentation) is 338.
    FVF_XYZ || FVF_NORMAL || FVF_DIFFUSE plus U and V are contained in the default settings.
    FVF_XYZ || FVF_NORMAL || FVF_DIFFUSE || FVF_TEX0 || FVF_TEX2 = 338
    An imported object will have these defaults applied   
    
    FVF_TEX0 can be added to any combination of other constants without altering the total.
    FVF integer values can be acquired using Matrix1 Get Limb FVF() or DKAVM's DK Get Object FVF().
    FVF values can be set as integer using Convert Object FVF.  Most values represent unique combinations
    (except for the addition of FVF_TEX0), but some do not.  If either FVF_XYZB5 or FVF_TEX0 is set using
    Convert Object FVF with the integer value, the object is converted to the 338 default, as shown by checking
    with the plugin tools mentioned above.  The value of 14 can be returned by multple combinations.  In these two
    cases (and others which may not represent unique values), the values should be explicated using the hex values.   
        
    http://doc.51windows.net/directx9_sdk/graphics/reference/d3d/constants/D3DFVF.htm
    http://www.codesampler.com/d3dbook/chapter_05/chapter_05.htm
    http://www.flipcode.com/archives/Flexible_Vertex_Format_Generator.shtml
    http://www.vbforums.com/showthread.php?t=548692 (Combination code)
remend

dim fvf(20)
dim fvf$(20,2)

fvf(1) = FVF_XYZ
fvf(2) = FVF_XYZRHW
fvf(3) = FVF_XYZB1
fvf(4) = FVF_XYZB2
fvf(5) = FVF_XYZB3
fvf(6) = FVF_XYZB4
fvf(7) = FVF_XYZB5
fvf(8) = FVF_NORMAL
fvf(9) = FVF_PSIZE
fvf(10) = FVF_DIFFUSE
fvf(11) = FVF_SPECULAR
fvf(12) = FVF_TEX0
fvf(13) = FVF_TEX1
fvf(14) = FVF_TEX2
fvf(15) = FVF_TEX3
fvf(16) = FVF_TEX4
fvf(17) = FVF_TEX5
fvf(18) = FVF_TEX6
fvf(19) = FVF_TEX7
fvf(20) = FVF_TEX8

fvf$(1,1) = "FVF_XYZ" 
fvf$(2,1) = "FVF_XYZRHW"
fvf$(3,1) = "FVF_XYZB1"
fvf$(4,1) = "FVF_XYZB2"
fvf$(5,1) = "FVF_XYZB3"
fvf$(6,1) = "FVF_XYZB4"
fvf$(7,1) = "FVF_XYZB5"
fvf$(8,1) = "FVF_NORMAL"
fvf$(9,1) = "FVF_PSIZE"
fvf$(10,1) = "FVF_DIFFUSE"
fvf$(11,1) = "FVF_SPECULAR"
fvf$(12,1) = "FVF_TEX0"
fvf$(13,1) = "FVF_TEX1"
fvf$(14,1) = "FVF_TEX2"
fvf$(15,1) = "FVF_TEX3"
fvf$(16,1) = "FVF_TEX4"
fvf$(17,1) = "FVF_TEX5"
fvf$(18,1) = "FVF_TEX6"
fvf$(19,1) = "FVF_TEX7"
fvf$(20,1) = "FVF_TEX8"

fvf$(1,2) = "0x002"
fvf$(2,2) = "0x004"
fvf$(3,2) = "0x006"
fvf$(4,2) = "0x008"
fvf$(5,2) = "0x00a"
fvf$(6,2) = "0x00c"
fvf$(7,2) = "0x00e"
fvf$(8,2) = "0x010"
fvf$(9,2) = "0x020"
fvf$(10,2) = "0x040"
fvf$(11,2) = "0x080"
fvf$(12,2) = "0x000"
fvf$(13,2) = "0x100"
fvf$(14,2) = "0x200"
fvf$(15,2) = "0x300"
fvf$(16,2) = "0x400"
fvf$(17,2) = "0x500"
fvf$(18,2) = "0x600"
fvf$(19,2) = "0x700"
fvf$(20,2) = "0x800"

calc = 0
calc$ = ""
r$ = ""
rsz = 1
dim rs$(rsz)
rs$ = clear_stringarray(rs$,rsz)
combos = 0
do    
    cls 
    set cursor 0,0
    print "FVF format value calculator"
    print
    for i=1 to 20
        pad$ = ") "
        if i < 10 then pad$ = pad$ + " "        
        fvfv$ = str$(fvf(i))
        if len(fvfv$) = 1 then fvfv$ = fvfv$ + "   "
        if len(fvfv$) = 2 then fvfv$ = fvfv$ + "  "
        if len(fvfv$) = 3 then fvfv$ = fvfv$ + " "        
        print i,pad$,fvf$(i,2)," = ",fvfv$," = ",fvf$(i,1)
    next i
    
    print 
    if calc > 0 then print calc$," = ",calc 
    if r$ <> "" then print r$   
    if combos > 0 then print "Checked ",combos," different combinations. " 
    
    for i=1 to rsz
        if rs$(i) <> "" then print rs$(i)
    next i
    print
    
    print "Enter 0 (zero) to clear calculation"
    print "Enter 21 to break an FVF value into its constants"
    input "Enter an index from FVF list to calculate (1-20) ",calcE
    r$ = "" : combos = 0 : rs$ = clear_stringarray(rs$,rsz)
    if calcE > 0 and calcE < 21
        calc = calc || fvf(calcE)
        if calc$ <> "" then calc$ = calc$ + " || "
        calc$ = calc$ + fvf$(calcE,1)
    endif
    if calcE = 0 then calc = 0 : calc$ = ""
    
    if calcE = 21
        input "Enter an FVF value to calculate (divisible by 2) ",calcE
        if calcE = 0 then rs$(1) = fvf$(12,1) + " = 0" `Special case.      
        if calcE > 0 and calcE mod 2 = 0
            input "Enter maximum number of returns (1-20) ",rsz
            if rsz > 20 or rsz < 1 then rsz = 1
            undim rs$()
            dim rs$(rsz)
            calc = 0 : calc$ = ""
            gosub _calculate_fvf
        endif               
    endif    
loop
end

_calculate_fvf:
    count = 0 : y = array count(fvf())
    rs$ = clear_stringarray(rs$,rsz)    
    for x=0 to (2 ^ y)-1             
         r$ = dec_to_bin(x,y)
         inc combos         
         if left$(r$,2) <> "11" and (mid$(r$,2) + mid$(r$,8) <> "11") 
             for i=1 to len(r$)
                 t$ = mid$(r$,i)
                 if t$ = "1"
                     calc = calc || fvf(i)
                     if calc$ <> "" then calc$ = calc$ + " || "
                     calc$ = calc$ + fvf$(i,1)                                                  
                 endif                           
             next i             
             if calc = calcE
                 inc count,1
                 rs$(count) = calc$ + " = " + str$(calc)
                 calc = 0 : calc$ = "" : r$ = ""                
                 if count = rsz then return                            
             endif    
             calc = 0 : calc$ = "" 
         endif       
    next x   
    if rs$(1) = "" then r$ = "No results found."    
return

function dec_to_bin(x,s)    
    r$ = ""
    for v=0 to s-1
        if x and (2 ^ v)
            r$ = r$ + "1"
        else
            r$ = r$ + "0"
        endif
    next v
endfunction r$

function clear_stringarray(rs$,rsz)
    for i=1 to rsz
        rs$(i) = ""
    next i
endfunction rs$

+ Code Snippet

` FVF constants - with value returned by Matrix1 plugin Get FVF Limb    
#constant FVF_XYZ = 0x002 `2 untransformed vertex - discludes FVF_XYZRHW
#constant FVF_XYZRHW = 0x004 `4 tranformed vertex - discludes FVFXYZ and FVF_NORMAL
#constant FVF_XYZB1 = 0x006 `6 position data and weighting values for vertex blending matrix calcs
#constant FVF_XYZB2 = 0x008 `8
#constant FVF_XYZB3 = 0x00a `10
#constant FVF_XYZB4 = 0x00c `12
#constant FVF_XYZB5 = 0x00e `338
#constant FVF_NORMAL = 0x010 `16 vertex normal
#constant FVF_PSIZE = 0x020 `32 vertex point size
#constant FVF_DIFFUSE = 0x040 `64 vertex diffuse color
#constant FVF_SPECULAR = 0x080 `128 vertex specular color
#constant FVF_TEX0 = 0x000 `338 Number of texture coordinate sets for vertex, 1 - 8
#constant FVF_TEX1 = 0x100 `256
#constant FVF_TEX2 = 0x200 `512
#constant FVF_TEX3 = 0x300 `768
#constant FVF_TEX4 = 0x400 `1024
#constant FVF_TEX5 = 0x500 `1280
#constant FVF_TEX6 = 0x600 `1536
#constant FVF_TEX7 = 0x700 `1792
#constant FVF_TEX8 = 0x800 `2048

remstart
    (Cagedrei's FVF notes)
    FVF value is an integer, calculated by specifying FVF constants combined using the bitwise or operation.
    The FVF value of the the DBP primitives is 274.
    The "default" FVF (covered in Make Memblock From Mesh documentation) is 338.
    FVF_XYZ || FVF_NORMAL || FVF_DIFFUSE plus U and V are contained in the default settings.
    FVF_XYZ || FVF_NORMAL || FVF_DIFFUSE || FVF_TEX0 || FVF_TEX2 = 338
    An imported object will have these defaults applied   
    
    FVF_TEX0 can be added to any combination of other constants without altering the total.
    FVF integer values can be acquired using Matrix1 Get Limb FVF() or DKAVM's DK Get Object FVF().
    FVF values can be set as integer using Convert Object FVF.  Most values represent unique combinations
    (except for the addition of FVF_TEX0), but some do not.  If either FVF_XYZB5 or FVF_TEX0 is set using
    Convert Object FVF with the integer value, the object is converted to the 338 default, as shown by checking
    with the plugin tools mentioned above.  The value of 14 can be returned by multple combinations.  In these two
    cases (and others which may not represent unique values), the values should be explicated using the hex values.   
        
    http://doc.51windows.net/directx9_sdk/graphics/reference/d3d/constants/D3DFVF.htm
    http://www.codesampler.com/d3dbook/chapter_05/chapter_05.htm
    http://www.flipcode.com/archives/Flexible_Vertex_Format_Generator.shtml
    http://www.vbforums.com/showthread.php?t=548692 (Combination code)
remend

dim fvf(20)
dim fvf$(20,2)

fvf(1) = FVF_XYZ
fvf(2) = FVF_XYZRHW
fvf(3) = FVF_XYZB1
fvf(4) = FVF_XYZB2
fvf(5) = FVF_XYZB3
fvf(6) = FVF_XYZB4
fvf(7) = FVF_XYZB5
fvf(8) = FVF_NORMAL
fvf(9) = FVF_PSIZE
fvf(10) = FVF_DIFFUSE
fvf(11) = FVF_SPECULAR
fvf(12) = FVF_TEX0
fvf(13) = FVF_TEX1
fvf(14) = FVF_TEX2
fvf(15) = FVF_TEX3
fvf(16) = FVF_TEX4
fvf(17) = FVF_TEX5
fvf(18) = FVF_TEX6
fvf(19) = FVF_TEX7
fvf(20) = FVF_TEX8

fvf$(1,1) = "FVF_XYZ" 
fvf$(2,1) = "FVF_XYZRHW"
fvf$(3,1) = "FVF_XYZB1"
fvf$(4,1) = "FVF_XYZB2"
fvf$(5,1) = "FVF_XYZB3"
fvf$(6,1) = "FVF_XYZB4"
fvf$(7,1) = "FVF_XYZB5"
fvf$(8,1) = "FVF_NORMAL"
fvf$(9,1) = "FVF_PSIZE"
fvf$(10,1) = "FVF_DIFFUSE"
fvf$(11,1) = "FVF_SPECULAR"
fvf$(12,1) = "FVF_TEX0"
fvf$(13,1) = "FVF_TEX1"
fvf$(14,1) = "FVF_TEX2"
fvf$(15,1) = "FVF_TEX3"
fvf$(16,1) = "FVF_TEX4"
fvf$(17,1) = "FVF_TEX5"
fvf$(18,1) = "FVF_TEX6"
fvf$(19,1) = "FVF_TEX7"
fvf$(20,1) = "FVF_TEX8"

fvf$(1,2) = "0x002"
fvf$(2,2) = "0x004"
fvf$(3,2) = "0x006"
fvf$(4,2) = "0x008"
fvf$(5,2) = "0x00a"
fvf$(6,2) = "0x00c"
fvf$(7,2) = "0x00e"
fvf$(8,2) = "0x010"
fvf$(9,2) = "0x020"
fvf$(10,2) = "0x040"
fvf$(11,2) = "0x080"
fvf$(12,2) = "0x000"
fvf$(13,2) = "0x100"
fvf$(14,2) = "0x200"
fvf$(15,2) = "0x300"
fvf$(16,2) = "0x400"
fvf$(17,2) = "0x500"
fvf$(18,2) = "0x600"
fvf$(19,2) = "0x700"
fvf$(20,2) = "0x800"

calc = 0
calc$ = ""
r$ = ""
rsz = 1
dim rs$(rsz)
rs$ = clear_stringarray(rs$,rsz)
combos = 0
do    
    cls 
    set cursor 0,0
    print "FVF format value calculator"
    print
    for i=1 to 20
        pad$ = ") "
        if i < 10 then pad$ = pad$ + " "        
        fvfv$ = str$(fvf(i))
        if len(fvfv$) = 1 then fvfv$ = fvfv$ + "   "
        if len(fvfv$) = 2 then fvfv$ = fvfv$ + "  "
        if len(fvfv$) = 3 then fvfv$ = fvfv$ + " "        
        print i,pad$,fvf$(i,2)," = ",fvfv$," = ",fvf$(i,1)
    next i
    
    print 
    if calc > 0 then print calc$," = ",calc 
    if r$ <> "" then print r$   
    if combos > 0 then print "Checked ",combos," different combinations. " 
    
    for i=1 to rsz
        if rs$(i) <> "" then print rs$(i)
    next i
    print
    
    print "Enter 0 (zero) to clear calculation"
    print "Enter 21 to break an FVF value into its constants"
    input "Enter an index from FVF list to calculate (1-20) ",calcE
    r$ = "" : combos = 0 : rs$ = clear_stringarray(rs$,rsz)
    if calcE > 0 and calcE < 21
        calc = calc || fvf(calcE)
        if calc$ <> "" then calc$ = calc$ + " || "
        calc$ = calc$ + fvf$(calcE,1)
    endif
    if calcE = 0 then calc = 0 : calc$ = ""
    
    if calcE = 21
        input "Enter an FVF value to calculate (divisible by 2) ",calcE
        if calcE = 0 then rs$(1) = fvf$(12,1) + " = 0" `Special case.      
        if calcE > 0 and calcE mod 2 = 0
            input "Enter maximum number of returns (1-20) ",rsz
            if rsz > 20 or rsz < 1 then rsz = 1
            undim rs$()
            dim rs$(rsz)
            calc = 0 : calc$ = ""
            gosub _calculate_fvf
        endif               
    endif    
loop
end

_calculate_fvf:
    count = 0 : y = array count(fvf())
    rs$ = clear_stringarray(rs$,rsz)    
    for x=0 to (2 ^ y)-1             
         r$ = dec_to_bin(x,y)
         inc combos         
         if left$(r$,2) <> "11" and (mid$(r$,2) + mid$(r$,8) <> "11") 
             for i=1 to len(r$)
                 t$ = mid$(r$,i)
                 if t$ = "1"
                     calc = calc || fvf(i)
                     if calc$ <> "" then calc$ = calc$ + " || "
                     calc$ = calc$ + fvf$(i,1)                                                  
                 endif                           
             next i             
             if calc = calcE
                 inc count,1
                 rs$(count) = calc$ + " = " + str$(calc)
                 calc = 0 : calc$ = "" : r$ = ""                
                 if count = rsz then return                            
             endif    
             calc = 0 : calc$ = "" 
         endif       
    next x   
    if rs$(1) = "" then r$ = "No results found."    
return

function dec_to_bin(x,s)    
    r$ = ""
    for v=0 to s-1
        if x and (2 ^ v)
            r$ = r$ + "1"
        else
            r$ = r$ + "0"
        endif
    next v
endfunction r$

function clear_stringarray(rs$,rsz)
    for i=1 to rsz
        rs$(i) = ""
    next i
endfunction rs$

Back to top

Profile PM Email

Sorry your browser is not supported!

Newcomers DBPro Corner / Magic FVF code to fix shader woes