Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Editor specific ‘require’ behaviour?

in Questions Posts: 198

Hi All,

This is probably best addressed directly to @Simeon to be honest but figured any discussion could be useful to others.

I’ve been making good progress on a minimal implementation of the luarocks package manager for use in Codea but have been struggling with some strange behaviour around the require() function in the editor.

This is the section of code I’m working with (the rest isn’t easily available as a single project yet):

local vfs = VFS("luarocks")

local function rock_loader(data)
    return function()
        return load(data)()
    end
end

local function rock_searcher(module)

    local search_paths = {
        "/share/lua/" .. module .. ".lua",
        "/share/lua/" .. module:gsub("%.", "/") .. ".lua",
        "/share/lua/" .. module .. "/init.lua"
    }

    -- Search the potential paths until we find a valid file
    for _,path in ipairs(search_paths) do
        local f = vfs:readFile(path)
        if f then
            return rock_loader(f)
        end
    end

    if module == "MessagePack" then
----------------Only works with this present-----------------
-------------------------------------------------------------
        return rock_loader('return { "I\'m never actually used!" } ')
-------------------------------------------------------------
    end

    -- We found no rock modules
    return nil
end

table.insert(package.searchers, 2, rock_searcher)



local ok, res = pcall(require, "MessagePack")
if ok then
    print("pcall 'require' success:", res)
else
    print("pcall 'require' fail:", res)
end



local res = require("MessagePack")
if res then
    print("Plain 'require' success:", res)
else
    print("Plain 'require' fail:", res)
end

The call to require wrapped in a pcall works just fine and it correctly finds the file and loads it without needing the if module == "MessagePack" then workaround.

The plain require call however, trips up immediately (without ever launching the project) without the workaround present making me wonder if something in the editor’s error checking is causing it?

Without the workaround I just see module ‘MessagePack’ not found in the editor on the local res = require("MessagePack") line. The most puzzling thing is that when the workaround is in place, it’s never actually used at all and the plain require correctly loads the intended file instead.

This has been causing me some real headaches just to get to this point so if anyone has any ideas I’d really appreciate it!

Cheers,
Steppers

Comments

  • edited July 29 Posts: 198

    It could have something to do with the way Codea executes code outside of the setup() & draw() functions.

    print("I print twice")
    
    function setup()
        print(“I print once”)
    end
    
    function draw()
        background(40, 40, 50)
    end
    

    In this case I’d expect the print() at the top to only be executed once, however when launching from the editor the string prints twice. When tapping the restart button after launching it the first time it’s only printed once.

    Is this due to some error checking being done before the project actually launches?

    Note that the initial issue with the require() function is not present when executing from within setup.

  • edited July 29 Posts: 198

    I’ve managed to narrow down the issue to the readText function when it’s used outside of setup() & draw() functions.

    Here’s a minimal repro case:

    local data = readText(asset.Main)
    
    if data == nil then
        -- Errors in the editor here when uncommented as the readText call above fails
        -- error(tostring(data))
    end
    
    function setup()
        print("Data from outside setup():\n" .. tostring(data):sub(1, 32) .. "...")
    
        data = readText(asset.Main)
        print("Data from inside setup():\n" .. tostring(data):sub(1, 32) .. "...")
    end
    

    It seems to be related to the 2 execution passes over the script before setup() is called. The first of which seems to cause readText to fail to read the file and return nil instead for some reason. Is it not fully initialised during this first pass @Simeon?

    And wrapping it in pcall means the file reads successfully somehow:

    local ok, data = pcall(readText, asset.Main)
    if ok then
        print(tostring(data):sub(1, 32) .. "...")
    else
        error(tostring(data))
    end
    
    function setup()
        print("Data from outside setup():\n" .. tostring(data):sub(1, 32) .. "...")
    
        data = readText(asset.Main)
        print("Data from inside setup():\n" .. tostring(data):sub(1, 32) .. "...")
    end
    
  • Posts: 172
    This might be a useless suggestion but what happens when you move the exterior code to a different file than Main?
  • Posts: 198

    @skar No change. All of stuff I’m working on at the moment has it across multiple files anyway and it’s the same issue.

  • Posts: 172
    One thing I think of is that maybe the function readText is not defined on the first pass
  • Posts: 198

    I’m fairly certain it’s defined though otherwise I’d get another error about calling a nil value iirc :/ Just seems to be silently failing.

  • dave1707dave1707 Mod
    Posts: 9,444

    @Steppers I’m not seeing the issue you describe for the nil readText. If I do a readText before setup() and 1 in setup(), and print each read, I print all 3. I don’t get the nil result. A restart results in 2 prints since the double pass isn’t required. The double pass when the project starts has been around for as long as I can remember. There’s very few commands that I use outside of a function because of the double pass at startup.

  • edited July 29 Posts: 198

    @dave1707 What about this one? The nil result only seems to be detectable if you generate an error manually.

    This should produce the error in the nil check block:

    local data = readText(asset.Main)
    
    if data == nil then
        -- Errors in the editor here when uncommented as the readText call above fails
        error(tostring(data))
    end
    
    print("Data while outside setup():\n" .. tostring(data):sub(1, 32) .. "...")
    
    function setup()
        print("Data from outside setup():\n" .. tostring(data):sub(1, 32) .. "...")
    
        data = readText(asset.Main)
        print("Data from inside setup():\n" .. tostring(data):sub(1, 32) .. "...")
    end
    
  • dave1707dave1707 Mod
    Posts: 9,444

    @Steppers You’re trying to read the Main tab with readText and it’s possible that the Main tab isn’t there yet to be read. Maybe it’s not there until the 2nd pass. Try running this code that calls the function “test” on the first pass. It causes an error because it doesn’t know anything about the function “test” yet.

    When I was doing the readTest before setup() and got all 3 prints instead of a nil, I was reading a text file that already existed.

    test()
    
    function setup()    
    end
    
    function test()
        print("test")
    end
    
  • edited July 29 Posts: 198

    @dave1707 What do you mean by ‘the Main tab isn’t there yet’?

    Even with this:

    local data = readText(asset.Main)
    
    error(tostring(data) .. " | " .. tostring(asset.Main))
    --print(tostring(data):sub(1, 32) .. "..." .. " | " .. tostring(asset.Main))
    

    I get an error in the editor reading nil | Asset Key meaning at least 1 pass is returning nil from readText. If I use the print line instead it never prints a nil value.

    That must be triggering the error in the very first pass (maybe even before the 2 we can see with print()) causing the project to never launch in the first place.

    This becomes a problem when trying to require modules outside of the setup() function as is common in much of standard Lua. My custom luarocks searcher for the require function relies on being able to read a json file to determine which file should be loaded but as the json is failing to load in the very first pass, require is then throwing an error meaning the project never launches at all.

  • dave1707dave1707 Mod
    Posts: 9,444

    @Steppers I’m wrong about the Main tab not being there on the first pass. If I run the below code, it prints the Main tab twice for the 2 passes, then once from setup.

    But then I’m confused about what’s really happening if you uncomment the 2nd print. That’s where your nil error shows up. But if it printed the Main tab twice the first time then “data” should have had something in it and not be nil.

    Maybe there’s more than 2 passes that we don’t know about and can’t test for.

    data = readText(asset.Main)
    print("===== before setup =====\n",data)
    --print("before setup=====\n"..data)
    
    function setup()
        data = readText(asset.Main)
        print("===== in setup =====\n"..data)
    end
    
  • Posts: 198

    Though many of the standalone issues I’ve mentioned above still stand It seems the main issue in my luarocks loading code was of my own making…

    I had an error call inside my virtual file system code that would trigger when we attempted to access a path that didn’t exist in the VFS. Now normally when running from the user’s code this would be just fine and the editor would display the relevant error from it, but in this case I suspect this was being triggered when an internal module middleclass and block was being required.

    Unfortunately, this error call was causing Codea to crash entirely so the runtime is clearly missing some error checking at that point in the project startup sequence.

    @Simeon Could be something to look into in future but I have everything working now that I was trying to do so it’s nothing urgent.

  • SimeonSimeon Admin Mod
    Posts: 5,705
    Do you have a sample that causes the crash, @Steppers
  • edited August 2 Posts: 198

    @Simeon Removing the last line should cause Codea to crash. Admittedly, raising an error in the custom module searcher does go against the lua 5.4 reference but It probably shouldn’t be crashing Codea.

    local function custom_searcher(module)
    
        if module == "HelloModule" then
            return function(module, data)
                return load("return 'Hello Codea'")()
            end, "nothing_important"
        end
    
        -- Lua 5.4 Reference states:
        -- 'Searchers should raise no errors and have no side effects in Lua'
        -- So I guess this could be classified as undefined behaviour.
        error("No module found in custom searcher")
    
        -- No module found
        return nil
    end
    
    table.insert(package.searchers, 2, custom_searcher)
    
    local str = require("HelloModule")
    
    function draw()
        background(40)
    end
    
    -- Codea crashes if I remove this.
    error(str)
    
  • SimeonSimeon Admin Mod
    Posts: 5,705

    Thank you @Steppers, fixed the crash.

    The error in the custom searcher was causing some custom Craft modules to throw exceptions when running a project. We catch the exceptions now, so this is fixed — in that it won't crash Codea. Though signalling an error in a custom searcher will likely cause other parts of Codea to stop working (as those modules won't be loaded)

  • Posts: 198
    @Simeon Amazing, thank you. Just good to not have Codea crash.

    I know the error shouldn't be there anyway and I've hopefully removed anything from my code that would trigger an error like that again so from my end we're all good.
  • SimeonSimeon Admin Mod
    Posts: 5,705

    @Steppers a new beta will be out with this fix

  • Posts: 1,295

    @Steppers sorry to not post anything helpful but once again Holy Shamoley you’re making cool things.

Sign In or Register to comment.