Improve non-interactive install where multiple packages match

[Thahun]

So far we used PECL and are now trying to replace it with Pie, but having some issues with automating it.

Just as a short background, due to our container structure we're at first building a base image and afterwards using that as a base to build more granular images for different purposes (e.g. app and cron). But they share the same PHP setup and to avoid redundant work and time, PHP is already installed in the base image. For now that was quite simple and we could simply use pecl install + docker-php-ext-enable to install and enable our PHP extensions like e.g. mongodb and redis. They were part of the base image and by that the setup was equal for all subsequent images. Next to that, build dependencies are deleted at the end of the build to keep the images smaller and the final containers are of course running as www-data so only the build has root permissions.
Of course we already use composer to install dependencies as well, so we already have them defined in the composer.json like

    "require": {
        "php": "~8.4.0",
        "ext-mongodb": "~2.1",
        "ext-redis": "~6.1",
        [...]
    }

Having the setup given, we tried to use Pie in a similar way and hit an issue with this non interactive approach.
Of course we could use Pie with hardcoded packages in the Dockerfile like:

pie install mongodb/mongodb-extension:~2.1
pie install phpredis/phpredis:~6.1
[...]

(Installing multiple extensions at once is not yet possible, but there's already a feature request for it: #466)

But we're more in favor of relying on the composer.json integration to avoid defining the extensions twice, cause for Composer they are anyways already defined. So just calling:

pie install [--allow-non-interactive-project-install]

And here's our issue, the resolution of ext-* to a package name is always triggering a prompt.
Even with --allow-non-interactive-project-install or --no-interaction the prompt remains (which is quite understandable, cause how should Pie be sure about which package is correct, except just using the suggested one automatically)
The only location we could determine where this mapping is persisted is the pie.lock in /root/.pie/php8.4_d2b6f37be78b1b629f281b53efb060b9/pie.lock but of course it's not designed to be modified.

So, the final question, is there any way to use Pie in a fully non interactive way, using the composer.json as a source?
At least I couldn't find anything in the documentation nor the command help.

The only working but unsafe way to achieve it, by automatically accepting option 1 of the prompts, seems to be currently

printf "%s\n" 1 | pie install --allow-non-interactive-project-install

[asgrim]

First a question for you:

For ext-mongodb and ext-redis, there should only be one option for either of those; so those two specific extensions shouldn't prompt when using --allow-non-interactive-project-install - could you check which specific extensions are being asked for in that case? To try to reproduce your issue, I built a Dockerfile with the following:

FROM ubuntu AS test_non_interactive
RUN apt-get update && apt-get install -y php unzip libbz2-dev gcc make autoconf bison re2c pkg-config libtool php-dev
COPY --from=build_pie_phar /app/pie.phar /usr/local/bin/pie
RUN mkdir /foo
WORKDIR /foo
RUN echo '{"require":{"php":"~8.3.0","ext-mongodb":"~2.1","ext-redis":"~6.1"}}' >> /foo/composer.json
RUN echo '{"_readme":["This file locks the dependencies of your project to a known state","Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies","This file is @generated automatically"],"content-hash":"201d95d4413cb1f999fba94555181bb0","packages":[],"packages-dev":[],"aliases":[],"minimum-stability":"stable","stability-flags":{},"prefer-stable":false,"prefer-lowest":false,"platform":{"php":"~8.3.0","ext-mongodb":"~2.1","ext-redis":"~6.1"},"platform-dev":{},"plugin-api-version":"2.6.0"}' >> /foo/composer.lock
RUN pie install --allow-non-interactive-project-install
RUN pie show

And this successfully installs the two missing extensions:

#19 [test_non_interactive 9/9] RUN pie show
#19 0.208 🥧 PHP Installer for Extensions (PIE) 1.3.5@a13873e, from The PHP Foundation
#19 0.322 You are running PHP 8.3.6
#19 0.340 Target PHP installation: 8.3.6 nts, on Linux/OSX/etc x86_64 (from /usr/bin/php8.3)
#19 0.391 Tip: to include extensions in this list that PIE does not manage, use the --all flag.
#19 0.574 
#19 0.574 Loaded PIE extensions:
#19 1.146   redis:6.3.0 (from 🥧 phpredis/phpredis:6.3.0 ✅)
#19 1.533   mongodb:2.1.4 (from 🥧 mongodb/mongodb-extension:2.1.4 ✅)
#19 DONE 1.5s

In this particular instance, it --allow-non-interactive-project-install should work just fine for you, I think?

However, yes; if there is more than one extension (maybe there is an extension not given in your example) that provides the ext-<extname> it will fail. In this circumstance, the issue is "how do we define that map" as you rightly point out. Using the pie.lock isn't going to work; as those instructions wouldn't exist in your "project" as such.

This is definitely something I'd like to improve on, but we need to find an effective way for PIE users to say "when I want ext-blah I mean extension foo/blah (as opposed to bar/blah or baz/blah-improved, etc.).

[Thahun]

Sorry for not providing the reproduceable example and wasting your time.

Cause now I discovered the issue that let us think we have an issue.

FROM ubuntu AS test_non_interactive
RUN apt-get update && apt-get install -y php unzip libbz2-dev gcc make autoconf bison re2c pkg-config libtool php-dev
COPY --from=ghcr.io/php/pie:bin /pie /usr/bin/pie
RUN mkdir /foo
WORKDIR /foo
RUN echo '{"require":{"php":"~8.3.0","ext-mongodb":"~2.1","ext-redis":"~6.1"}}' >> /foo/composer.json
RUN echo '{"_readme":["This file locks the dependencies of your project to a known state","Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies","This file is @generated automatically"],"content-hash":"201d95d4413cb1f999fba94555181bb0","packages":[],"packages-dev":[],"aliases":[],"minimum-stability":"stable","stability-flags":{},"prefer-stable":false,"prefer-lowest":false,"platform":{"php":"~8.3.0","ext-mongodb":"~2.1","ext-redis":"~6.1"},"platform-dev":{},"plugin-api-version":"2.6.0"}' >> /foo/composer.lock
WORKDIR /foo

docker build -t pie_test .
docker run -it pie_test

root@a220a1923045:/foo# pie install  --allow-non-interactive-project-install
🥧 PHP Installer for Extensions (PIE) 1.3.5, from The PHP Foundation
You are running PHP 8.3.6
Target PHP installation: 8.3.6 nts, on Linux/OSX/etc arm64 (from /usr/bin/php8.3)
Checking extensions for your project __root__ (path: /foo)
requires: ext-mongodb:~2.1 🚫 Missing

The following packages may be suitable, which would you like to install:
  [0] None
  [1] mongodb/mongodb-extension: MongoDB driver extension
 > 0
Okay I won't install anything for mongodb
requires: ext-redis:~6.1 🚫 Missing

The following packages may be suitable, which would you like to install:
  [0] None
  [1] phpredis/phpredis: A PHP extension for Redis
 > 0
Okay I won't install anything for redis

Finished checking extensions.

The whole issue is that the --allow-non-interactive-project-install option doesn't have any effect on an interactive shell, and by that the install command can't be manually tested. Or at least it behaves different and we didn't expect that. So we just thought it's not working as expected while infact it works on a real non-interactive shell, just not on an interactive shell.

Independet of that, yes it would be good to have some option to define a mapping of ext to package, cause pretty sure at some point there'll be extensions matching multiple packages. In best case this could be done directly in composer.json to avoid a second config, but even an additional config yaml and/or option for the install command would be fine from my point of view.

[asgrim]

No worries at all :) FWIW, you could use COMPOSER_NO_INTERACTION=1 pie install --allow-non-interactive-project-install to simulate a non-interactive shell (I think)

But yes, I would like to have some way of defining what extensions are picked, but I was hoping to shelve that problem for now 😆

[Thahun]

Yes, that works, I tried it. We just didn't have that in mind. Of course it's obvious when having the relation between Composer and Pie in mind that Composer environment variables should work as well, but maybe it's worth to add a hint to the documentation of --allow-non-interactive-project-install or even a general one if all Composer environment variables are useable in Pie as well.

Anyways, thanks for the help :)