Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Codea (Beta 15 & 16)

SimeonSimeon Admin Mod
edited January 2013 in Beta Posts: 5,364

This beta is starting to get closer to the release version.

Bindings

The main new feature is the "Bindings" tab in the Shader Lab. This should make writing shaders much more interesting.

The bindings tab is like a spreadsheet of your uniform variables. You can assign arbitrary Codea expressions to each uniform (such as CAMERA, or ElapsedTime, or CurrentTouch, and so on). They don't have to be simple, any Lua code will work as an expression.

The Shader Lab now includes a whole bunch of example shaders. I'm a bit unsure if I will be able to include certain shaders, because some were taken from places like Heroku, where the license is unclear. If you have any cool shaders to contribute, please send them to me and I will include them.

Issues carried over from the previous thread:

Tagged:

Comments

  • Posts: 2,161

    Here's code that demonstrates the problem I'm having with taking photos. The problem only shows with the rear-camera and only when the picture is taken. It also only happens in portrait mode (I don't know if it used to fail in landscape as well - I hadn't tested it before β15).


    -- Camera -- Use this function to perform your initial setup function setup() print("This example shows you how to use camera input") -- Create a switch for our camera source parameter.integer("CameraSource", 0, 1) -- We'll start by setting the source to the front camera cameraSource(CAMERA_FRONT) -- Get the width and height camWidth, camHeight = spriteSize( CAMERA ) end function checkCameraSource() -- Here we'll check our parameter and change sources accordingly if CameraSource == 0 then cameraSource( CAMERA_FRONT ) else cameraSource( CAMERA_BACK ) end camWidth, camHeight = spriteSize( CAMERA ) end -- This function gets called once every frame function draw() -- Configure our camera source checkCameraSource() -- This sets a dark background color background(40, 40, 50) -- This sets the line thickness strokeWidth(5) -- Draw the special CAMERA sprite sprite( CAMERA, WIDTH/2, HEIGHT/2, math.min( camWidth, WIDTH ) ) if img then sprite(img,100,100,200) end end function touched(touch) if touch.state == ENDED then img = image(CAMERA) end end
  • SimeonSimeon Admin Mod
    Posts: 5,364

    .@Andrew_Stacey thanks for the example, I've finally managed to fix this one (it was due to the rear camera returning padded data for portrait orientation).

    .@John has also fixed the setContext() viewport clipping issue, as well as bool and float array support for shaders.

    There are only a few remaining issues and we'll be able to submit 1.5 for everyone.

  • Posts: 2,161

    .@Simeon What's the status with the various SetContext issues? Will we get a z-level? What about the fact that drawing on to images seems to clip against the current screen (so drawing on to a portrait image when in landscape mode and vice versa)?

  • Posts: 2,161

    .@Simeon Whoops - cross-posting! That's great about the viewport clipping.

  • SimeonSimeon Admin Mod
    Posts: 5,364

    I'm unsure if we can respect the depth buffer in render-to-texture (i.e. setContext). It requires an OpenGL ES extension that appears to be available on most iPads, but I'm not sure how much work it will be to implement.

  • SimeonSimeon Admin Mod
    edited January 2013 Posts: 5,364

    Beta 16 is going up.

    Addresses setContext viewport bug and saving camera images from the rear camera in portrait orientations.

  • Jmv38Jmv38 Mod
    Posts: 3,295

    Hello @Simeon. Do you plan to repair the output select/copy in final version? This is useful to report error messages or anything else....

  • Posts: 1,255

    @Simeon Earlier you had proposed a "copy" button that would pick up the output contents and move them to the clipboard, since the new output doesn't support copy/paste. Any chance of that sneaking in?

  • Posts: 489

    Hello @Simeon. The odd behaviour I was having with my 'arc' code still persists with beta 1.5(16). I am using iOS 6.1 on an iPad 2.

  • edited January 2013 Posts: 489

    Pressing the 'Replay' button four or five times with the following code causes Codea to crash:

    -- Print Bug ?
    
    function setup()
        for k, v in pairs(_G) do
            print(k, v)
        end
    end
    
    function draw()
        background(0)
    end
    
  • SimeonSimeon Admin Mod
    Posts: 5,364

    .@mpilgrem that's an odd one, and hard to figure out. It only happens when my device is disconnected from the debugger, making it extremely hard to figure out what the crash is.

  • SimeonSimeon Admin Mod
    Posts: 5,364

    Has anyone tried the bindings feature? It would be great to get some feedback on it before release.

  • SimeonSimeon Admin Mod
    edited January 2013 Posts: 5,364

    .@mpilgrem edit: I've looked into this and it appears that rapidly creating and assigning shaders to mesh objects can cause the modelViewProjection matrix not to be set correctly. You can fix this for now by modifying your arc code to set it manually:

    --
    -- Arc
    --
     
    supportedOrientations(LANDSCAPE_ANY)
    function setup()
        parameter.number("width", 1, 100, 40)
        parameter.number("radius", 50, WIDTH / 2, 200)
        parameter.number("startAngle", -180, 180, -130)
        parameter.number("endAngle", -180, 180, 130)
        stroke(118, 223, 25)
    end
     
    function draw()
        background(64)
        strokeWidth(width)
        arc(WIDTH / 2, HEIGHT / 2, radius, startAngle, endAngle)
    end
     
    function arc(x, y, radius, a1, a2)
        local m = mesh()
        m:addRect(x, y, radius * 2, radius * 2)
        m.shader = shader("Documents:Arc")
        m.shader.size = (1 - strokeWidth()/radius) * 0.5
        m.shader.color = color(stroke())
        m.shader.modelViewProjection = modelMatrix() * viewMatrix() * projectionMatrix()
        m.shader.a1 = math.rad(a1)
        m.shader.a2 = math.rad(a2)
        m:draw()
    end
  • Posts: 1,255

    Anyone else having trouble getting 16? Despite several hours of trying, cold restarts, and trying different networks, I'm still getting "this application can't be downloaded right now."

  • Jmv38Jmv38 Mod
    Posts: 3,295

    I installed yesterday, it was ok.

  • Posts: 489

    Hello @Simeon. Thank you for the patch on 'Arc'.

    I've tried the bindings feature, and it seems to work as you have explained.

    I have one thought: it would be good to be able to copy of all the bindings into a 'snippet' of text that could then be pasted elsewhere (for example, on to the Forum). How about touching and holding on any of the Uniforms would bring up a pop up: 'Copy all'? To use my 'Arc' shader as an example, this could then create a snippet:

    float a1 = 0.0;
    float size = 0.3;
    ...
    mat4 modelViewProjection = modelMatrix() * viewMatrix() * projectionMatrix();
    

    that could be pasted elsewhere?

  • Posts: 489

    (1) I appreciate that the Shader Lab will be beta in Codea 1.5, but is there any possibility of being able to 'duplicate' or 'rename' Shaders in Documents/Your Shaders?

    (2) Given the icon for the 'Basic' pack of Shaders in the Lab, should the example shader 'Ellipse' be in the 'Basic' pack and not in the 'Patterns' pack?

    (3) If the mat4 modelViewProjection binding is not previously set, it would be good if the default binding first created was not matrix() but modelMatrix() * viewMatrix() * projectionMatrix()

  • SimeonSimeon Admin Mod
    Posts: 5,364

    .@mpilgrem thanks for your feedback. being able to copy all the bindings was exactly a feature I was thinking of — I had pushed it back until after 1.5, but given that you have brought it up independently I might be able to put it back in.

    There used to be a button called "Shader Info" that was going to present sample code of how to use the shader with your saved bindings assigned to uniforms, and the option to copy the Lua code to pasteboard.

    1. I think I might be able to fit that feature in, at least a duplicate function if not rename.

    2. Yes I had the same thought, the problem is the ellipse code is complex and I wanted dead simple shaders under "Basic". I just couldn't think of an appropriate icon.

    3. Good point, I will set its default value to that Codea expression.

  • Jmv38Jmv38 Mod
    edited January 2013 Posts: 3,295

    .@Simeon a very important feature is the ability to set a shader from the lua program. Can the binding be set from the program like you've done for the vertext and fragment shaders?

  • SimeonSimeon Admin Mod
    Posts: 5,364

    You have to set them yourself, when you use the shader. The Shader Lab sets the bindings every frame for all uniforms, but that would be slow and potentially unnecessary for most use cases. Like @mpilgrem mentioned, I will offer a way for you to copy the 'sample' binding code out of the Shader Lab.

  • Posts: 489

    In the Bindings editor, I have quickly found myself missing the ability to place the location of the cursor precisely.

  • SimeonSimeon Admin Mod
    Posts: 5,364

    .@mpilgrem I will be bringing the extended keyboard and autocomplete to the bindings editor, but not until 1.5.x. I miss it too.

  • Posts: 503

    Hi. In the recent releases you dont move to the search result when pressing on a result. Shouldnt it move to the result?

    I recently used the old Codea, what a difference! The help is so much better now. :)

  • SimeonSimeon Admin Mod
    Posts: 5,364

    .@tnlogy yes, it should. That's a bug I've had on my list for a while.

    Glad to hear the help is much better.

  • Hi,

    jumping in late to test the Beta, had some other things going on... current version really looks cool :-)

    I have tried an early beta and wanted to do a lighting shader, as thats most important for my project, but as I remember, there were no normals in the mesh. Now I can see them in the documentation, so I think they have been added meanwhile (or I have overlooked them before).

    Now my question is: Did anybody else already create a lighting shader? I searched the forum, but only found a post that somebody wanted to try. Otherwise I will try...

    I think there should be everything thats needed for it: The projection and modelview matrices and the vertex normals. The light position can be passed into the shader then. I'd like to replace my lua calculated lighting demo posted some time ago with a shader.

    Kilam.

  • Posts: 2,161

    .@KilamMalik It might be a bit basic for you, but my toroidal Game of Life shader had a lighting component.

  • Jmv38Jmv38 Mod
    Posts: 3,295

    .@KilamMalik i think the mesh.normal is just a predefined name for mesh attributes, but it is not computed by codea: you have to compute and assign the normals yourself. So there is not much new there.

  • SimeonSimeon Admin Mod
    Posts: 5,364

    Yes, you have to compute normals yourself. We could probably compute flat-facing normals per-triangle, though you'd probably want to overwrite them with smooth normals.

  • Posts: 503

    Here is an example reading an Obj file and using a light shader. You have to point out a texture, could not figure out any way in Codea to get data from an image to create a string. Are there any functionality for that, so that I can use image.data?

    https://gist.github.com/4690383

  • edited February 2013 Posts: 196

    edit:@tnlogy bet me to it heh :P I'll leave my example, since i don't play with the buffer functionality.

    @KilamMalik - It don't think anyone has posted a simple one

    In my example, you will notice the normals are handled properly by the mesh class, and properly used as vec3 attribute by the shader.

    I believe there was a plan in Codea to implement them, that's why they're handled properly ?

    local data = mesh()
    data.vertices = verts
    data.colors = colors
     -- you need to compute the normals, I use my blender to image exporter
    data.normals = normals
    
    data.shader = shader("Documents:celShading")
    data.shader.light = vec3(1,-1,1):normalize()
    
    // Vertex shader
    uniform mat4 modelViewProjection;
    uniform lowp vec3 light;
    
    attribute vec4 position;
    attribute vec4 color;
    attribute vec3 normal;
    
    varying lowp vec4 vColor;
    varying lowp float shade;
    void main()
    {
        vColor = normalize(modelViewProjection * vec4(normal,0));
        shade = dot(vColor.xyz, light);
        vColor = color;
    
        gl_Position = (modelViewProjection * position);
    }
    
    //fragment
    varying lowp vec4 vColor;
    varying lowp float shade;
    
    void main()
    {
        if (shade<.2)
            gl_FragColor = vec4(vColor.xyz * .3, 1.0);
        else
        if (shade<.4)
            gl_FragColor = vec4(vColor.xyz * .7, 1.0);
        else
            gl_FragColor = vColor;
    }
    

    Cheers

  • Posts: 503

    An irritating bug that I see from time to time is when press play one of the tabs turn red, but there is no indication where the error is. In earlier releases you could see the error in the panel, since it switched view. I suppose you are supposed to see the error message in the tab, but it only works sometimes.

    A future feature I would like is if say, the first 4 items in projects are the most recently used projects. :)

  • SimeonSimeon Admin Mod
    Posts: 5,364

    .@tnlogy if the tab highlights red then that indicates a syntax error usually — so the code shouldn't be able to run. Are you able to reproduce coding errors that can trigger a tab to highlight red with no error message?

  • Sometimes I get error messages but have to manually scroll to the line. Not nice when there are several hundred lines in the editor.

    Here's a reproduction of a truly missing message that seems to work always for me:

    • Go to Physics Lab example
    • In Main, add a single curly brace } in the first empty line
    • Switch to PhysicsDebugDraw tab
    • Hit run, the Main tab turns red but no message can be found
    • In Main, hit run again, message appears

    Premature analysis:

    • When in different tab, message gets lost
    • When in same tab, message appears but no auto-scroll to that line
  • SimeonSimeon Admin Mod
    Posts: 5,364

    Thank you @Codeslinger. I've changed this behaviour so that error messages no longer disappear when switching into a tab with errors, and the editor will automatically scroll to the first error in a tab that contains errors when opening it.

  • Jmv38Jmv38 Mod
    edited February 2013 Posts: 3,295

    .@tnlogy I sometimes get the red tab with no erro message. I 've figured out that 95% of times it is after modying an if.. then...else ... end statement and forgetting the end keyword. Then the compiler goes all the way to the end of the tab, which makes sense, and there it does not find the expected end keyword. So it knows the problem, but the error can be anywhere in the code. It does not make sense to show the last line in error, because the error is not there. So next time you get the red tab, check the last if.then.else.end you've modified.

  • Jmv38Jmv38 Mod
    Posts: 3,295

    .@Simeon Hi would it be possible to include to version 1.5 the ability to load images of any size? Like 5000x4000 for instance, even on ipad1? If you cannot display them because too big for shader and texture limits, it is ok because i just want to display a subpart of this image. The problem is not breaking the image into pieces once it is in codea, it is to open the jpeg file. Because i cannot access to the jpeg raw data with codea, i cannot write my own decoder. So can you do something for that? I am open to any compromize solution too, as long as i can read the big original jpeg or png image directly into codea...

  • edited February 2013 Posts: 196

    @Simeon - regarding bindings, it seems that shaders from previous versions show the wrong Codea Expression for the mat4 modelViewProjection (new shaders work fine).

    New shaders show "modelMatrix() * viewMatrix() * projectionMatrix()" while old shaders only show "matrix()".

    I suppose it's not really a problem if you think about it, but just letting you know in case there is an underlying issue.

  • SimeonSimeon Admin Mod
    Posts: 5,364

    .@Jmv38 a 5000x4000 would take up 76 MB of memory when loaded, an iPad 1 has 256 MB RAM. Once you account for the OS and apps, it usually has less than 100 MB free. I think loading such a large image on iPad 1 into system memory would cause the application to quit due to low memory.

    There are ways to load such images that use a tile-based renderer or loader, but that would take us a long time to implement, and certainly wouldn't be possible to include in 1.5 without significantly delaying it.

    .@Xavier thank you for the report. Shaders from previous versions never had bindings, so Codea simply generates a default value for a mat4 type, which is matrix(). This issue only effects beta users with old shaders — it won't be an issue in the release version.

  • Posts: 196

    @Simeon that's what I figured :P

  • SimeonSimeon Admin Mod
    Posts: 5,364

    I've removed modelViewProjection from the bindings editor in the next build. In addition, setting the modelViewProjection property on a shader object will cause a warning to be printed stating that Codea will overwrite the value.

    This fixes the bug that @mpilgrem experienced with his Arc code — however it means you can not set a custom modelViewProjection matrix. If you want to use a custom matrix in your vertex shader you will have to create a mat4 uniform with a different name (you can also delete the modelViewProjection uniform from your vertex shader if you're not using it — and instead use your own).

    In the future I'd like to change this behaviour so that modelViewProjection can be overwritten by the user, but if not set, use the current matrix. This won't make it for 1.5 though.

  • Jmv38Jmv38 Mod
    edited February 2013 Posts: 3,295

    Thanks for your answer about big images @Simeon. I understand your position. And what about just being able to read a part of binary file into a table of values? You probably already have the code available, so there is almost no dev. Is it one of the things forbiden by Apple?

  • SimeonSimeon Admin Mod
    Posts: 5,364

    You should be able to read binary files using the io.* package in Lua, it is unsandboxed in version 1.5.

  • Simeon/Jvm38/tnology/Xavier,

    thanks for your answers. Yes, I calculate the normals on my own, or better said, I import it from PLY. Until now I had to copy and paste a PLY into an editor tab, which could get tricky for large 3D models :-) But as I understand, 1.5 is unsandboxed, so I could read the PLY from file -> Very nice.

    These are the two programs where I calculate the lighting in LUA and want to replace with the shader:

    http://twolivesleft.com/Codea/Talk/discussion/1528/code-sample:-lighting-in-3d-meshes#Item_7

    http://twolivesleft.com/Codea/Talk/discussion/1723/3d-model-loading-from-blender-ply-and-suggestion:-need-to-load-local-files#Item_44

    Let's see how far I get with that :-)

    Kilam.

  • Jmv38Jmv38 Mod
    edited February 2013 Posts: 3,295

    .@Simeon this is great! I will try it soon then. Meanwhile i was looking into pythonista which has a lot of tools, and i found i could lauch a pythonista script from Codea with:

        openURL("pythonista://Clock?action=run") -- launch 'clock' script
    

    does codea has a 'url scheme' like pythonista? This way i would be able to launch a Codea script from pythonista, which means i can launch python from codea for some specifics tools and send back the results to codea... Wouldn't that be great? I have looked in the list of url scheme collected by python community, but did not find Codea in it.

  • SimeonSimeon Admin Mod
    Posts: 5,364

    I've been thinking about adding a URL scheme to Codea. You mention that results can be sent back, how does that work?

  • Jmv38Jmv38 Mod
    edited February 2013 Posts: 3,295

    They describe the following in their doc: i can send data to the script with:

    Basics
    Open the app:
    
    Use pythonista:// without any additional parameters to open the app without doing anything else.
    
    Open a script for editing:
    
    Use pythonista://MyScript for opening one of your scripts in the editor. The name of the script can contain the .py file extension, but it doesn’t have to.
    
    Run a script from your library:
    
    Use pythonista://MyScript?action=run for running a script that is in your library. This only works if no script is already running when the app is started.
    
    Note You can only open or run scripts that are already in your library. It is not possible to run arbitrary code or create new scripts using the URL scheme.
    Command-line Arguments
    When using the action=run parameter (see above), you can pass command-line arguments to the script in two ways:
    
    Method 1: One args parameter
    
    Example: pythonista://MyScript?action=run&args=foo%20bar
    
    When you pass a single string with the args URL parameter, the string is split by spaces (the space is encoded as %20 here) and sys.argv would be [<script_path>, 'foo', 'bar'] in this example.
    
    If you want to pass arguments that contain spaces, you have to enclose them in double quotes (as you would in a classic shell).
    
    This is basically the same as running a script by long-pressing the run button.
    
    Method 2: Multiple argv parameters
    
    Example: pythonista://MyScript?action=run&argv=foo&argv=bar
    
    Using multiple arguments that are all named argv, you don’t have to worry about quoting arguments that contain spaces. Each of the parameters corresponds to one element in sys.argv.
    
    If the URL contains at least one argv parameter, args is ignored.
    
    Launching Other Apps
    If you want to go in the opposite direction, and open an app from Pythonista, you can do this with the webbrowser module.
    
    For example, to open TweetBot, you could use the following code:
    
    CopyOpen in Editor
    import webbrowser
    webbrowser.open('tweetbot://')
    

    so my idea is to write in pythonista webbrowser.open("codea://myscript.codea?myVeryLongString=str") at the end of my python script to pass data as strings back to codea.
    Maybe it even already works with http.request fom Codea, since python url scheme is just like a web browers, and you have nothing to change? But then i dont know how to do it yet.

This discussion has been closed.