Apply shader to Craft model?

iam3o5am

Is this possible? Can my own mesh w/shader be added as a Craft model?

Thanks.

piinthesky

see https://codea.io/talk/discussion/8806/converting-meshes-to-models

iam3o5am

No talk there of shaders (from what I read). I don’t see where on a Craft model I can apply shaders.

Simeon

@iam3o5am

You can use them like regular Craft materials (though I think this has been a bit improved in the latest Codea beta — let me know if you want to join that)

entity.material = craft.material("Documents:MyShadeShader")
entity.material.someShadeParameter = 10
entity.material.someShadeColor = color(255, 0, 0)

Just drag and drop your Shade material or copy-and-paste it from Shade into the Codea documents directory in "On My iPad -> Codea" on your filesystem. Then they will show up in the material picker in Codea.

dave1707

@Simeon Thanks. That was posted once before but you show it here before I was able to find it.

iam3o5am

@Simeon Yes please, I’d love to join beta testing. Let me know what you need from me.

Simeon

@iam3o5am dm me your Apple ID email

iam3o5am

Any way to define the shader (as strings) directly in code, on the craft material, just as we do on mesh objects?

Simeon

@iam3o5am there might be, probably more a question for @john though you can just bundle shaders directly in your projects now (I.e “Project:MyShader”)

John

@iam3o5am You can define shaders in code yourself if you want. What you need to do is create a special shader definition table and add that to the shader system so it can be compiled.

To simplify some things there is a surface shader system where you can define functions to determine vertex and fragment shader outputs with simplified parameters. You can make stuff physically based or unlit. You can also write a shader from scratch but that's more complicated.

Here's an example project:

-- Custom Shader

customShaderUnlit = {
    name = "Custom Unlit",

    options =
    {
        USE_COLOR = { true },
    },

    properties =
    {
        map = {"texture2D", nil}
    },

    pass =
    {
        base = "Surface",

        blendMode = "disabled",
        depthWrite = true,
        depthFunc = "lessEqual",
        renderQueue = "solid",
        colorMask = {"rgba"},
        cullFace = "back",

        vertex =
        [[
            void vertex(inout Vertex v, out Input o)
            {
            }
        ]],

        surface =
        [[
            uniform sampler2D map;

            void surface(in Input IN, inout SurfaceOutput o)
            {                
                o.diffuse = texture(map, IN.uv * 10.0).rgb;
                o.emissive = 1.0;                                
                o.emission = vec3(0.0, 0.0, 0.0);
            }
        ]]
    }
}

customShaderPhysical = {
    name = "Custom Physical",

    options =
    {
        USE_COLOR = { true },
        USE_LIGHTING = { true },
        STANDARD = { true },
        PHYSICAL = { true },
        ENVMAP_TYPE_CUBE = { true },
        ENVMAP_MODE_REFLECTION = { true },
        USE_ENVMAP = { false, {"envMap"} },
    },

    properties =
    {
        envMap = { "cubeTexture", "nil" },
        envMapIntensity = { "float", "0.75" },
        refactionRatio = { "float", "0.5" },
    },

    pass =
    {
        base = "Surface",

        blendMode = "disabled",
        depthWrite = true,
        depthFunc = "lessEqual",
        renderQueue = "solid",
        colorMask = {"rgba"},
        cullFace = "back",

        vertex =
        [[
            void vertex(inout Vertex v, out Input o)
            {
            }
        ]],

        surface =
        [[
            void surface(in Input IN, inout SurfaceOutput o)
            {
                o.diffuse = vec3(1.0, 1.0, 1.0);
                o.roughness = 0.32468;
                o.metalness = 0.0;
                o.emission = vec3(0.0, 0.0, 0.0);
                o.emissive = 1.0;
                o.opacity = 1.0;
                o.occlusion = 1.0;
            }
        ]]
    }
}


function setup()
    -- Create a new craft scene
    scene = craft.scene()

    -- Add the shader definitions to the rendering system
    craft.shader.add(customShaderUnlit)
    craft.shader.add(customShaderPhysical)    

    -- Create a new entity
    local e1 = scene:entity()
    e1.model = craft.model("Primitives:Sphere")
    e1.material = craft.material("Custom Unlit")    
    e1.material.map = readImage("Blocks:Wood")
    e1.x = 2
    e1.z = 10

    local e2 = scene:entity()
    e2.model = craft.model("Primitives:Sphere")
    e2.material = craft.material("Custom Physical")    
    e2.x = -2
    e2.z = 10
end

function update(dt)
    -- Update the scene (physics, transforms etc)
    scene:update(dt)
end

-- Called automatically by codea 
function draw()
    update(DeltaTime)

    -- Draw the scene
    scene:draw()    
end

Unlike the old mesh system you need to specify the names of any uniforms you want to access as property materials. This allows you to specify default values as well. Options allow for different shader variants, some of which are special reserved names for things like lighting, and others can be defined and linked to properties. This lets you write shaders with lots of options, like the Standard material, which comes with craft.

I've also attached zipped up versions of these basic shaders that you could edit and then add as bundles to your projects or the documents folder.

John

Oh yeah in case you were wondering what is in Vertex, Input and SurfaceOutput, these are all structs which pass data between parts of the shader programs to determine the final appearance:

struct Vertex
{
    vec3 position;
    vec3 normal;
    vec2 uv;
#ifdef USE_COLOR
    vec3 color;
#endif
};

struct Input
{
    vec3 worldPosition;
    vec3 worldNormal;
    vec3 viewPosition;
    vec3 normal;
    vec2 uv;
#ifdef USE_COLOR
    vec3 color;
#endif
#ifdef USE_TANGENTS
    vec3 tangent;
    vec3 bitangent;
#endif
#ifdef USE_TANGENT_VIEW_DIR
    vec3 tangentViewDir;
#endif
};

Unlit SurfaceOutput:

struct SurfaceOutput
{
    vec3 diffuse;
    vec3 emission;
    float emissive;
    float opacity;
#ifdef BEHIND
    vec3 behind;
#endif
};

Physical SurfaceOutput

struct SurfaceOutput
{
    vec3 diffuse;
    vec3 normal;
    vec3 emission;
    float emissive;
    float roughness;
    float metalness;
    float occlusion;
    float opacity;
#ifdef BEHIND
    vec3 behind;
#endif
};

Note that some things are inside of #ifdef macros, which are used in conjunction with the options in the shader definition to turn certain features on and off.

iam3o5am

Thanks @John - that’s very helpful. Between these examples and looking at the rest of the Shade examples, I’ve got plenty to go on.

A final question, then, is this: Is there a way to directly use the GLSL vertex and fragment shader code that can be used now on a mesh object? Basically, is there any way to use a Codea mesh shader (say from an older Codea project), as is, in Craft, or should I buckle up and just start mastering Shade? The latter option is exciting anyway, and potentially easier to create advanced materials, but just wondering if using direct GLSL is still a possibility. As a concrete example, could I, via Shade, set GLSL options, such as #GL_OES_standard_derivatives, etc.?

Thanks! Enough on this thread, just wanted to get a handle on where its all going and what possibilities there are.

Simeon

@iam3o5am I would probably hold off on too much GLSL. One of the main reasons we started building Shade was to get away from GLSL because Apple has deprecated OpenGL and will be removing it from iOS in a couple years.

The plan with Shade is to have it generate Metal and Unity shaders. At the moment its GLSL code-gen is a temporary measure while Codea still runs on OpenGL.

iam3o5am

Will the Codea API change considerably once OpenGL is gone? (Going now to read up on this (maybe not so new?) news from Apple...)

John

@iam3o5am The API itself will remain very similar (some key things might change though). Things related to mesh, shader and the craft material system will undergo the most changes. The biggest one being the switch to Metal shader syntax.
Metal as a graphics API is much more modern and easier to work with than OpenGL but aside from the shaders this isn't going to be exposed to the end user all that much.

piinthesky

@John will that mean that shadertoy shaders will no longer work (arghhh)?

John

Unless we add some kind of GLSL to Metal cross compiler it’s a bit iffy.

UberGoober

@Simeon is your code above, to convert shaders into materials, still functional?

I’m trying:

        robot.material = craft.material(asset.builtin.Basic.Invert)

...which seems to be what you did in the pasted code, and it doesn’t seem to be working.

dave1707

@Simeon I hope that doesn’t get rid of the vertex and fragment shader coding. I like being able to use that for increased speed when updating the screen.

West

@UberGoober I get an error when trying to us the built in shaders, but it works ok if I drag across a shader from shade:

viewer.mode=FULLSCREEN

function setup()
    scene = craft.scene()
    scene.camera.position=vec3(0,0,-8)

    cameraComponent = scene.camera:get(craft.camera)
    cameraComponent.hdr = true
    cameraComponent.colorTextureEnabled = true

  material = craft.material(asset.builtin.Basic.Invert)


    ground = scene:entity()
    ground.model = craft.model.plane(vec2(1,1))
    ground.position = vec3(0,0,0)
    ground.eulerAngles=vec3(-90,0,0)
    ground.material = material
end

function update(dt)
    scene:update(dt)
end

function draw()
    update(DeltaTime)
    scene:draw()
end

ERROR IS

Main:11: Failed to load shader: /private/var/containers/Bundle/Application/E5A6F631-66D0-4A92-B9A7-3ABD61559B67/Codea.app/Assets/Basic.assets/Invert.harder stack trace back” [c]: in field ‘material’

piinthesky

did anyone manage to implement a glsl shader directly on a craft model using inline code not via Shade? @John provided something earlier in this discussion but it was not clear to me how to use that. @Simeon, @John could you be kind enough to provide the full code for a simple example.