Support proxy definition for fetch operation by using custom http client using proxy env var or flags to command line

[matthewdwood82]

Bug description

I am attempting to install a quarto extension for section bibliographies into a Quarto Book project using PowerShell.
https://github.com/pandoc-ext/section-bibliographies#quarto

When I attempt to do so, I receive this error

My project has my company's http_proxy and https_proxy defined in the .Rprofile

It is not clear from quarto help how the user would verify quarto can see these proxy settings.

Checklist

• Please include a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
• Please format your issue so it is easier for us to read the bug report.
• Please document the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
• Please document the operating system you're running. If on Linux, please provide the specific distribution.
• Please provide the output of quarto check so we know which version of quarto and its dependencies you're running.

[cderv]

For reference this was also ask on Community: https://community.rstudio.com/t/chapter-references-in-quarto-pdf/166151

I believe for quarto add (replacing quarto install) and using the Deno Fetch API, setting the env var should be supported : HTTP_PROXY, HTTPS_PROXY and NO_PROXY

Is it not working ?

[matthewdwood82]

It is not, using either quarto add pandoc-ext/section-bibliographies or the old quarto install extension pandoc-ext/section-bibliographies

I'm new to Deno and TS, is there a good example of a setter using quarto run and a deno or ts script? The current quarto documentation provides a nice example of referencing a script file, but a layer of detail down on setting env vars would be helpful in my case.

Thanks!

[cderv]

@dragonstyle the download method we use for quarto add seems to be the Fetch API from DENO. From doc, I was expecting the env var to be respected, but does not seems the case.

Do you think it should work ?

Otherwise, we may need to add some flags for proxy to set our call to download method with proxy values

[dragonstyle]

It looks to me like those env variables are respected by DENO for import resolution, but not natively by fetch (without a little code from us). Seem like we can create a custom http client for use in fetch:

const client = Deno.createHttpClient({ proxy: { url: "http://myproxy.com:8080" } });
const response = await fetch("https://myserver.com", { client });

But we aren't using that in our core downloader :(. We probably need to support proxy flags for the commands + ideally we'd sniff those same env variables...

[matthewdwood82]

Thanks @dragonstyle for the nice example. Will give it a try.

[matthewdwood82]

It looks to me like those env variables are respected by DENO for import resolution, but not natively by fetch (without a little code from us). Seem like we can create a custom http client for use in fetch:

const client = Deno.createHttpClient({ proxy: { url: "http://myproxy.com:8080" } });
const response = await fetch("https://myserver.com", { client });

But we aren't using that in our core downloader :(. We probably need to support proxy flags for the commands + ideally we'd sniff those same env variables...

Sorry I didn't think to ask sooner, should https://myserver.com be a reference to my company's domain, or the GIT repo from which I'm trying to pull the extension?

[cderv]

@matthewdwood82 I am not sure if you want to test or just install the extension

For the latter you can download the extension as an archive file, and install it from the local downloaded file
https://quarto.org/docs/extensions/distributing.html#archive-files

But we aren't using that in our core downloader :(. We probably need to support proxy flags for the commands + ideally we'd sniff those same env variables...

@dragonstyle this is what I found also - I was not sure so thanks for confirming. It seems the view docs I found about env var with Fetch were wrong. Good to know !

I'll rename the issue accordingly to what we need to do

[matthewdwood82]

@matthewdwood82 I am not sure if you want to test or just install the extension

For the latter you can download the extension as an archive file, and install it from the local downloaded file https://quarto.org/docs/extensions/distributing.html#archive-files

But we aren't using that in our core downloader :(. We probably need to support proxy flags for the commands + ideally we'd sniff those same env variables...

@dragonstyle this is what I found also - I was not sure so thanks for confirming. It seems the view docs I found about env var with Fetch were wrong. Good to know !

I'll rename the issue accordingly to what we need to do

Thanks @cderv , more interested in installing extension, using an archive sounds like a fine way to stay out of server and client config shenanigans.

Reporting back shortly...

[matthewdwood82]

That worked! We can close this one, pending any future enhancement the dev team here decides to make wrt quarto and proxies.

Thanks @cderv & @dragonstyle for your help!