C Libraries

[ByTheBookGames]

Still trying to figure out how the Lunalua setup for SMBX compares to normal use of Lua elsewhere. I've been able to get luasocket working on SMBX by tweaking some core lua files like lockdown and main, fetching html information like IP addresses etc., but I'm currently working on getting luafilesystem operational and not having as much luck. (Right now all this is for an auto update script that will automatically download new files listed online as part of newer versions of an episode, though I know there are security reasons for not wanting to set things up this way.) The gist of what I'm wondering is:

- Why does the "require" function only load .lua files, and not also .dll files?
- Where is main.lua finding files like "jit.util"? Are these a part of the compiled LunaDll.dll which loads Lunalua onto SMBX on boot?
- Is it possible to load a c library within a regular lua file in an episode? How would that work?

Thanks for taking the time to help, it's very appreciated!

[Hoeloe]

None of this is a good idea. By making these changes you're not going to be able to do this and also share anything you make with it. The purpose of lockdown.lua is, as the name suggests, to lock down the things LunaLua can load in order to prevent people from building malware in SMBX (which is extremely possible with access to the filesystem and luasocket, it's very easy to do). I believe LunaLua can absolutely require .dll files (SMBX2 used to have one in its core files), but from what I understand, Lua can't just load any .dll you like, it requires it to be built from Lua bytecode. As for jit.util, I'm pretty sure that's part of the Lua core, since that's part of the JIT compiler.

I am pretty certain it is not possible to load a C library using Lua, regular or otherwise. We can access C code from LunaDLL, but not arbitrary libraries by my understanding of how things are set up.

[ByTheBookGames]

By making these changes you're not going to be able to do this and also share anything you make with it. The purpose of lockdown.lua is, as the name suggests, to lock down the things LunaLua can load in order to prevent people from building malware in SMBX (which is extremely possible with access to the filesystem and luasocket, it's very easy to do).

I understand this and figured as much. The changes made in my version of lockdown.lua are for private use. For example, sharing an SMBX project in development with other individuals (not publically shared) while utilizing an auto-update feature as mentioned allows for a more efficient way of synchronizing newer versions and changes before I feel it's polished enough for release. That hypothetical public release version would not include changes to lockdown or the update system... It's mostly just an independent hobby, and I'm asking for the sake of understanding SMBX's Lunalua potential, not to steal anyone's credit card information. (Though I completely understand the security concern.)

I will mention that while luasocket's functionality is limited without changes to lockdown.lua, much of it is still useful regardless, like connecting two devices for the exchange of data for online multiplayer. (I did need to change main.lua to require socket.core at 7 different places first.) While I've gotten some form of this to work through LAN, I needed away to connect through global ip (similar to how Minecraft servers work, still within security-conscious reason), and the following code worked for me, after manually downloading luasocket as a folder within an SMBX episode and tweaking its .lua src scripts' loaded APIs/"require":

Code: Select all

-- Slight modification of Thijs's code, giving an error of your global ip using ipinfo.io:
local socket = require("socket.core")
local http = require("luasocket/src/http")
local ltn12 = require("luasocket/src/ltn12")
local json = require("ext/lunajson")  

local resp,code,headers,status = http.request("http://ipinfo.io/json")
  if tostring(code) == "permission denied" then error("Something is preventing this game from accessing the internet--most likely your firewall. To play online, change your firewall's settings to allow smbx.exe.") end
	if code ~= 200 then return nil, "Failed fetching external IP; "..tostring(status) end
  local data, err = json.decode(resp)
  if not data then
    return nil, "Failed fetching external IP; "..tostring(err)
  end
  if data.ip then
    error(tostring(data.ip))
  else
    return nil, "Failed fetching external IP; no ip field was returned"
  end

_____________________________________________________________________________________________________________________

I believe LunaLua can absolutely require .dll files (SMBX2 used to have one in its core files), but from what I understand, Lua can't just load any .dll you like, it requires it to be built from Lua bytecode. As for jit.util, I'm pretty sure that's part of the Lua core, since that's part of the JIT compiler. I am pretty certain it is not possible to load a C library using Lua, regular or otherwise. We can access C code from LunaDLL, but not arbitrary libraries by my understanding of how things are set up.

Thanks for your answers, that's very helpful. After seeing your comments, I reinvestigated the following Lua documentation pages:

https://www.lua.org/pil/8.2.html
https://www.lua.org/pil/26.2.html

I'm still a bit confused, because it claims on the one hand: "Unlike packages written in Lua... C packages need to be loaded and linked with an application before use." But then on the other hand: "Lua provides all the functionality of dynamic linking in a single function, called loadlib."

I tried running a .dll using loadlib to test this, and my code looked like this:

Code: Select all

    local path = Misc.resolveDirectory("luafilesystem/src").."/lfs.dll"
    local f = assert(package.loadlib(path, "luaopen_lfs"))
    f()

However, the error message I got back was that "loadlib" was a nil function. I tried it as package.loadlib and just loadlib, and neither worked. The second documentation page does state "The most convenient way to do it is with the dynamic linking facility, if your Lua interpreter supports this facility," so perhaps SMBX does not support dynamic linking like this, which is why the loadlib function won't work within a Lua file, because it has to be pre-loaded elsewhere? Tutorials such as this one (https://blog.devgenius.io/how-to-add-yo ... fd246f0fa8) seem to indicate that .so and .dll file types should be loadable through "require" same as .lua files, though of course, I'm a novice at all this so I'm probably misunderstanding Lua or am mistaken.

[Emral]

while utilizing an auto-update feature as mentioned allows for a more efficient way of synchronizing newer versions and changes before I feel it's polished enough for release

Why not just use some form of version control like git?

[Hoeloe]

while utilizing an auto-update feature as mentioned allows for a more efficient way of synchronizing newer versions and changes before I feel it's polished enough for release

Why not just use some form of version control like git?

This is absolutely the sensible option. Automatic updates are not a good idea. Version control allows you to remotely update changes and synchronize, without running into issues with multiple people editing the same files at the same time and having to merge the results. It also allows you to roll back changes and diagnose where specific issues might have come from. I strongly suggest, instead of doing all of this work, you just look at Github or Gitlab and use that.

As for loadlib, it depends a lot which version of Lua you're talking about. The Lua version SMBX uses is LuaJIT, which I believe is based on Lua 5.1. For SMBX purposes, "C packages need to be loaded and linked with an application before use" is correct, and this is done by LunaDLL (this is linked into the SMBX exe by the launcher).

[ByTheBookGames]

Why not just use some form of version control like git?

Version control allows you to remotely update changes and synchronize, without running into issues with multiple people editing the same files at the same time and having to merge the results. It also allows you to roll back changes and diagnose where specific issues might have come from.

That definitely sounds simpler, but isn't the system you're referring to primarily used for collaborating with other developers? What about "playtesters," so to speak? Wouldn't github still require users to manually check and download recent builds, whether through code or checking online? And isn't SMBX's integrated episode update checker just as good for that purpose? Maybe there's a simple way that git automatically runs version checks and downloads (the synchronizing you referred to?), and I'm just not aware of it. I use github in web development through Visual Studio, but it's primarily the classic git add ., git commit -m, git push stuff.

As for loadlib, it depends a lot which version of Lua you're talking about. The Lua version SMBX uses is LuaJIT, which I believe is based on Lua 5.1. For SMBX purposes, "C packages need to be loaded and linked with an application before use" is correct, and this is done by LunaDLL (this is linked into the SMBX exe by the launcher).

Interesting! I didn't know about LuaJIT. Is there a .c version--or just uncompiled version--of LunaDLL accessible? If you'd prefer not to share this I understand.

[Emral]

I always invite playtesters to my repositories because automatic updates suck and setting up launcher-versioning in an episode before release is inefficient. If you have a discord server with your playtesters you can set up a webhook to post new commits automatically so playtesters always know when something new can be pulled.

[ByTheBookGames]

I always invite playtesters to my repositories because automatic updates suck and setting up launcher-versioning in an episode before release is inefficient. If you have a discord server with your playtesters you can set up a webhook to post new commits automatically so playtesters always know when something new can be pulled.

Ha, good to know. Thanks for sharing your insights.

By the way, looking into LuaJIT, there is actually a simple way of using C code within SMBX. In main.lua, I took "local" off of "ffi = require("ffi")" (should be around line 240). This made ffi global, and therefore usable within any episode's lua scripts. This solved the problem I was looking to luafilesystem for help with: creating a folder/directory. Here's what it looked like for me:

Code: Select all

local C = ffi.C
ffi.cdef[[
	bool CreateDirectoryA(const char *path, void *lpSecurityAttributes);		
]]
C.CreateDirectoryA("PathForNewDirectory", nil)

Worked like a charm. Not sure if you would consider globalizing ffi to be a security risk or not, but just thought I'd share in case it's useful. I personally find it very handy to be able to access C code within an episode, if I'm ever limited with Lua.

Cf: https://luajit.org/ext_ffi.html

[Emral]

As soon as people start packaging C DLLs into their episodes I'll never feel safe downloading an episode again. The lua environment is locked down to the episode folder for a reason.

[ByTheBookGames]

As soon as people start packaging C DLLs into their episodes I'll never feel safe downloading an episode again. The lua environment is locked down to the episode folder for a reason.

Well, didn't mean to raise so many alarms. My apologies. Thank you for your timely interactions and helpful insights in this forum.