building from source task

[pxp9]

the purpose from this task is to make easy to test Elixir AtomVM with main branch code from the AtomVM repository for ESP32.

this task requires idf.py installed or docker for compiling the AtomVM from source.

feel free to comment any issue in the implementation as well.

Thank you for your attention.

Workflow with custom build

mix atomvm.esp32.build --atomvm-path ./_build/atomvm_source/AtomVM/ --use-docker --chip esp32s3

mix atomvm.esp32.install --image ./_build/atomvm_source/AtomVM/src/platforms/esp32/build/atomvm-esp32s3.img

Using ESP-IDF Docker container

[UncleGrumpy]

This is a very cool feature! One that has been on my todo list to add to the atomvm_rebar3_plugin for a while. I work much more in Erlang, so my Elixir is not that great, but I did spot a few problems, and have some suggestions on improving usability.

[UncleGrumpy]

It might also be good to consider allowing a ESP-IDF docker image. I suspect many more Elixir developers will already have docker installed (or at least be more familiar with it than ESP-IDF), so it would be extra cool to pull in a docker image of ESP-IDF if they don't already have it installed.

[pxp9]

Hey @UncleGrumpy , when i try to build AtomVM , the build fails because we are using the the stat struct but the header <sys/stat.h> is missing, is there something i need to add in the args of cmake to build it correctly or is this a bug in AtomVM main branch build ?

[913/1054] Building C object esp-idf/avm_sys/CMakeFiles/__idf_avm_sys.dir/sys.c.obj
  FAILED: [code=1] esp-idf/avm_sys/CMakeFiles/__idf_avm_sys.dir/sys.c.obj

  /home/pxp9/Programming/try_nerves/atom_vm_firmware/esp32_s3/_build/atomvm_source/AtomVM/src/platforms/esp32/components/avm_sys/sys.c: In function 'sys_open_avm_from_file':

  /home/pxp9/Programming/try_nerves/atom_vm_firmware/esp32_s3/_build/atomvm_source/AtomVM/src/platforms/esp32/components/avm_sys/sys.c:413:21: error: storage size of 'file_stats' isn't known
    413 |         struct stat file_stats;
        |                     ^~~~~~~~~~

  /home/pxp9/Programming/try_nerves/atom_vm_firmware/esp32_s3/_build/atomvm_source/AtomVM/src/platforms/esp32/components/avm_sys/sys.c:414:22: error: implicit declaration of function 'stat' [-Wimplicit-function-declaration]
    414 |         if (UNLIKELY(stat(path, &file_stats) < 0)) {
        |                      ^~~~

  /home/pxp9/Programming/try_nerves/atom_vm_firmware/esp32_s3/_build/atomvm_source/AtomVM/src/platforms/esp32/components/avm_sys/sys.c:413:21: warning: unused variable 'file_stats' [-Wunused-variable]
    413 |         struct stat file_stats;
        |                     ^~~~~~~~~~

  Build stopped:
  ninja: build stopped: subcommand failed.

[pxp9]

Added docker feat based on this

https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-guides/tools/idf-docker-image.html#building-a-project-with-cmake

[UncleGrumpy]

Hey @UncleGrumpy , when i try to build AtomVM , the build fails because we are using the the stat struct but the header <sys/stat.h> is missing, is there something i need to add in the args of cmake to build it correctly or is this a bug in AtomVM main branch build ?

This is not something I have seen before, and I have been working almost exclusively with main branch for at least the last few months.

Was this testing a docker build, or did this happen with an installed ESP-IDF? It has been a few ESP-IDF releases since I last used an esp-idf docker image, although we do use them in the AtomVM CI, so I don’t think that would be the problem.

[pxp9]

@UncleGrumpy , this issue from the missing header happened with both toolchains. Esp idf installed and docker

[UncleGrumpy]

Interesting. Which ESP-IDF version? I think I am a point release behind locally (still using 5.5.1). I will try updating my toolchain this evening and see if I can reproduce the problem, I believe the AtomVM CI may also need to be bumped to 5.5.2, I will look into that too. This may be a new issue that you were the first to discover. I think you should open an issue in the AtomVM repo, and be sure to include Host OS, and ESP-IDF version in the report, along with the error and any other details that might be relevant.

[pxp9]

I tried with the last ESP version 6.X and then I tried with 5.4.1 which afaik is the last supported by AtomVM but not sure. It happens the same with both versions.

[UncleGrumpy]

I tried with the last ESP version 6.X and then I tried with 5.4.1 which afaik is the last supported by AtomVM but not sure.

I have not tried 6.0 yet, but I believe a PR was recently merged to allow ESP-IDF 6.0 builds. ESP-IDF 5.5.1 is definitely working for main branch, we might not have updated the main branch docs to reflect this, that is something we will need to make sure is up to date before a 0.7.0 release of AtomVM. Since you are hitting this error with ESP-IDF 5.4.1 and 6.0 then I don’t think just switching to 5.5.x will be any different.

[pxp9]

Okay @UncleGrumpy , today I will post an issue in AtomVM with this error and how to reproduce it.

Meanwhile could you review the code and see if everything is fine ?

[pxp9]

Hey @UncleGrumpy , I tried 4 times to compile the AtomVM cloning each time just in case and it seems to be fixed.

this command but with each version

mix atomvm.esp32.build --use-docker --chip esp32s3 --idf-version latest

1. I tried with v5.4.1 ESP-IDF container and it worked
2. I tried with v5.5.1 ESP-IDF container and it worked
3. I tried with latest (v6.1) ESP-IDF container and it did not work
4. I tried with (the tag release-v6.0) ESP-IDF container and it did not work

it seems i had something weird in the cache or it has been fixed.

by default --use-docker uses the version v5.4.1

[UncleGrumpy]

Hey @UncleGrumpy , I tried 4 times to compile the AtomVM cloning each time just in case and it seems to be fixed.

this command but with each version

mix atomvm.esp32.build --use-docker --chip esp32s3 --idf-version latest

1. I tried with v5.4.1 ESP-IDF container and it worked
2. I tried with v5.5.1 ESP-IDF container and it worked
3. I tried with latest (v6.1) ESP-IDF container and it did not work
4. I tried with (the tag release-v6.0) ESP-IDF container and it did not work

it seems i had something weird in the cache or it has been fixed.

by default --use-docker uses the version v5.4.1

That is great! I would not necessarily expect any of the 6.x IDF versions to work at the moment. I think AtomVM itself will need some updating. If it works with stable releases from 5.1 (or 5.2) to 5.5 that is all that would be expected at the moment. The AtomVM main branch uses 5.5.x for release builds so that should probably be the default here too. I believe 5.5.2 is the latest, we still need to bump that in the AtomVM CI, which I believe is still using 5.5.1. There shouldn’t be any breakage between these versions since the point releases are usually just bug fixes.

[UncleGrumpy]

Meanwhile could you review the code and see if everything is fine ?

I will take another look this evening, but I think you will get some much more useful feedback when some of devs that work more in Elixir have a chance to review this, especially @bettio and @petermm.

[UncleGrumpy]

yup the result of the docker build is like this, but the native AtomVM build it is working. i believe it is bug with the docker flow in the tool, but it is hard to find.

I discovered the problem with the docker container. I am working on a fix for the AtomVM repo that will change the configuration of the mkimage.sh and mkimage.config. The problem is that the paths to the binary files are configured by cmake in the container and then need to be executed in the host environment to use OTP to execute the escript. I can change the mkimage.sh script to find the esp32/build dir at runtime (rather than have it set by cmake) and pass that directly to the escript to use as the base for finding the partition binaries.

We should fix this there so users, other tools, and IDEs can use docker builds.

[pxp9]

Amazing I had a mini fix which was patching the paths for the container build but it seems that is not working.

[pxp9]

Happy new year ! 🎆

[UncleGrumpy]

You can try using this fix atomvm/AtomVM#2052 for the AtomVM repo and see if that fixes your problems with the docker builds.

[pxp9]

@UncleGrumpy just tried your changes about that PR and the docker build is working !!!

@petermm , could you try to build the atomvm with the task with the custom mbed tls path ??

[UncleGrumpy]

@UncleGrumpy just tried your changes about that PR and the docker build is working !!!

Great! That is not a complete fix, though. I targeted that fix to main, I probably should have done that on release-0.6 branch (maybe we can back port it, or I can submit a separate fix for release-0.6)… but that still won’t cover previous release tags, like v0.6.6. There really isn’t a way to push the fix back to cover those, so you may need to work out the path rewrite for those older tags, or we can just say this feature is supported for main branch (and release-0.6 branch if we apply the fix there too) and tagged releases starting from 0.7.0 (or 0.6.7 if we back port… and do a 0.6.7 release).

Another option might be to use git and cherry-pick the fix on top of whatever older checkout that the user might building from.

[pxp9]

@UncleGrumpy just tried your changes about that PR and the docker build is working !!!

Great! That is not a complete fix, though. I targeted that fix to main, I probably should have done that on release-0.6 branch (maybe we can back port it, or I can submit a separate fix for release-0.6)… but that still won’t cover previous release tags, like v0.6.6. There really isn’t a way to push the fix back to cover those, so you may need to work out the path rewrite for those older tags, or we can just say this feature is supported for main branch (and release-0.6 branch if we apply the fix there too) and tagged releases starting from 0.7.0 (or 0.6.7 if we back port… and do a 0.6.7 release).

Another option might be to use git and cherry-pick the fix on top of whatever older checkout that the user might building from.

I believe this feature is oriented to people who want to build and experiment with newer releases, anyways, if they want to build 0.6.6 they can always make the build with Esp IDF installed in the host instead of docker build.

I will document that docker build is supported since the moment you merged that PR in main branch.

I think adding this feature for older AtomVM builds doesn't give much more value.

[UncleGrumpy]

I agree. If they want older tagged releases they can just install the official release download. It will be devs wanting to test the newest features that will be using this task.

[pxp9]

I agree. If they want older tagged releases they can just install the official release download. It will be devs wanting to test the newest features that will be using this task.

Amazing, I will commit soon with the docs comment and addressing Peter comment as well.

[pxp9]

@UncleGrumpy , added Note about docker usage.

add docker usage note, and address build generic_unix options passing

i was thinking we can add a guide of how to install the ESP-IDF in nix flake since it is a reproduceable environment is cool. (for telling the users who want to build old releases how to make them).

[UncleGrumpy]

i was thinking we can add a guide of how to install the ESP-IDF in nix flake since it is a reproduceable environment is cool. (for telling the users who want to build old releases how to make them).

This would be great, but this repo would not be the best place for that. I would suggest either submitting “Tutorial” here:
https://atomvm.org/tutorials/

This can be done from the atomvm/atomvm.org repo. I would add a new page and link to it from the Tutorials page.

Or you could add a new section to the end of the esp32 build instructions here:
https://doc.atomvm.org/main/build-instructions.html#building-for-esp32

Which is generated from atomvm/AtomVM/doc/src/build-instructions.md.

[pxp9]

i was thinking we can add a guide of how to install the ESP-IDF in nix flake since it is a reproduceable environment is cool. (for telling the users who want to build old releases how to make them).

This would be great, but this repo would not be the best place for that. I would suggest either submitting “Tutorial” here: https://atomvm.org/tutorials/

This can be done from the atomvm/atomvm.org repo. I would add a new page and link to it from the Tutorials page.

Or you could add a new section to the end of the esp32 build instructions here: https://doc.atomvm.org/main/build-instructions.html#building-for-esp32

Which is generated from atomvm/AtomVM/doc/src/build-instructions.md.

@UncleGrumpy

i believe tutorials section is better, do you think this is ready to be merged ?

[pxp9]

@bettio , could you review please ?

[petermm]

In the interest of calling a spade a spade - I suggest calling this cmake_prefix as that is what it is..

[pxp9]

The reason is called mbed_prefix is because it edits in the cmake for building mbed_tls the cmake prefix

Maybe rename to mbedtls_cmake_prefix ?

Another reason it was called mbed_prefix is bc the user interfacing option, this variable holds the path for the custom mbed_tls. That's why the current name is not bad.

[UncleGrumpy]

I agree, I think medtls should be in the name. At some point in the future someone may need to also expose an option to set the path to zlib, which might be installed in a completely different location.

[petermm]

yeah I understand , but I believe this would work for multiple prefixes:

--cmake-prefix "/usr/local/opt/mbedtls@3;/usr/local/opt/zlib"

I think we should also expose build flags.. at least in future PR..

[UncleGrumpy]

That might be better for a future PR. I can see a lot of future enhancements for this task, but we should get a minimal task that can build for all supported host platforms first.

[pxp9]

This makes the user interface task attached to cmake knowhow , if you are an Elixir user , you probably don't know how to build the cmake_prefix, I think we should simplify this with each lib option (AtomVM does not have infinite libs which link with cmake). Then internally we will make a function which takes all these options and builds the proper cmake_prefix.

This is also nice bc if AtomVM decides at some point to change the build system (for example, building with Zig compiler which maybe in some years is not something crazy), the user API for this task will remain more less the same.

[pxp9]

I think this API is better

mix esp32.build --mbedtls /path/mbedtls --zlib /path/zlib 

Than

mix --cmake-prefix /path/mbedtls;/path/zlib

[pxp9]

I think this API is better

mix esp32.build --mbedtls /path/mbedtls --zlib /path/zlib 

Than

mix --cmake-prefix /path/mbedtls;/path/zlib

[petermm]

ok, let's stick with that direction then..

[UncleGrumpy]

Thats fine, I think as long as there is a brief explanation of the option and its purpose in the help and docs it doesn’t matter much either way.

[petermm]

I'm pretty sure we don't want to populate avm_deps in the context of esp32 - eg. a "hello world" my_project yields a 1,3 Mb my_project.avm file with the avm_deps folder populated..

an empty avm_deps yields a 768 bytes my_project.avm file.. massive difference.

we may need this functionality for other platforms like stm32 or pico - this also highlights how we might want the atomvmlib building to be a separate mix task (or module at least) (esp32.build can still call it for identical functionality, but split code up..)