Axios and proxies for Node.js marketing tools

Marketing-technology and e-commerce teams build plenty of small Node.js services: a dashboard that pulls public competitor prices, a checker that confirms campaign URLs still resolve, a feed that drops regional data into a BI tool. Axios is a common HTTP client in those projects, and sending it through a proxy lets each request come from the market you need.

There is one trap to avoid. Axios's own proxy option does not tunnel HTTPS requests reliably, and nearly every site a business tool talks to uses HTTPS. The dependable answer is a proxy agent. This guide sets one up with https-proxy-agent, then adds a shared client, rotation, retries and SOCKS5.

Where Axios and a proxy fit in a marketing stack

• Price and availability feeds for products your team tracks in several countries.
• Campaign link checks that confirm each URL answers correctly in its target market.
• Regional data for reporting, collected from public pages and APIs on a schedule.

Install and check the credentials

npm install axios https-proxy-agent

Your order supplies four values, shown below as HOST, PORT, USER and PASS. Check them from a terminal before writing any Node code:

curl --proxy http://USER:PASS@HOST:PORT https://api.ipify.org

Use a proxy agent, not the proxy field

Rather than filling in Axios's proxy field, create an agent from the proxy URL and pass that to Axios. The agent manages the CONNECT tunnel for HTTPS sites, login included:

const axios = require("axios");
const { HttpsProxyAgent } = require("https-proxy-agent");

const proxyUrl = "http://USER:PASS@HOST:PORT";
const agent = new HttpsProxyAgent(proxyUrl);

(async () => {
  const res = await axios.get("https://api.ipify.org?format=json", {
    httpAgent: agent,
    httpsAgent: agent,
    timeout: 20000
  });
  console.log(res.data);
})();

Assigning the same agent to httpAgent and httpsAgent covers plain and encrypted sites alike. If the response shows the proxy's address, the tunnel is working. This is the pattern to standardise on across your services.

Why the built-in option causes trouble

Here is the native setup, so you can recognise it in older code:

const res = await axios.get("http://httpbin.org/ip", {
  proxy: {
    protocol: "http",
    host: "HOST",
    port: Number("PORT"),
    auth: { username: "USER", password: "PASS" }
  }
});

It can work against plain HTTP addresses, but HTTPS tunnelling through it is unreliable across Node and Axios versions, and the errors it produces are hard to read. Since almost every modern site is HTTPS, the agent approach is the one we recommend.

One shared client for the whole service

When most of a service's requests use the same proxy, configure an Axios instance once and import it wherever it is needed. Proxy settings then live in one place:

const axios = require("axios");
const { HttpsProxyAgent } = require("https-proxy-agent");

const agent = new HttpsProxyAgent("http://USER:PASS@HOST:PORT");

const client = axios.create({
  httpAgent: agent,
  httpsAgent: agent,
  timeout: 20000,
  headers: { "User-Agent": "Mozilla/5.0 (compatible; MyScraper/1.0)" }
});

client.get("https://example.com").then(r => console.log(r.status));

The User-Agent in this example names the tool openly, which is a courteous habit for any business crawler. In production, load the proxy URL from an environment variable or your secrets manager instead of writing credentials into the code.

Spreading requests across endpoints

If the order provides several sticky endpoints, create an agent for each and alternate between them, so no single address handles everything:

const axios = require("axios");
const { HttpsProxyAgent } = require("https-proxy-agent");

const urls = [
  "http://USER:PASS@HOST:PORT1",
  "http://USER:PASS@HOST:PORT2",
  "http://USER:PASS@HOST:PORT3"
];
const agents = urls.map(u => new HttpsProxyAgent(u));
let i = 0;

async function fetch(url) {
  const agent = agents[i++ % agents.length];
  return axios.get(url, { httpAgent: agent, httpsAgent: agent, timeout: 20000 });
}

fetch("https://api.ipify.org?format=json").then(r => console.log(r.data));

With one rotating gateway you need just one agent, because each request through it leaves from a new address anyway.

Retries with growing pauses

At any real volume, some requests will meet a dropped connection or a 429 response. A small wrapper retries with longer waits each time:

async function getWithRetry(url, opts, tries = 4) {
  for (let attempt = 1; attempt <= tries; attempt++) {
    try {
      return await axios.get(url, opts);
    } catch (err) {
      const status = err.response && err.response.status;
      if (attempt === tries || (status && status < 500 && status !== 429)) throw err;
      await new Promise(r => setTimeout(r, attempt * 1500));
    }
  }
}

It tries again after network errors, 429s and 5xx responses, and stops immediately on client errors such as 404, which another attempt would not fix.

SOCKS5 option

For SOCKS5, use a different agent package:

npm install socks-proxy-agent

const { SocksProxyAgent } = require("socks-proxy-agent");
const agent = new SocksProxyAgent("socks5://USER:PASS@HOST:PORT");
// then pass it as httpAgent and httpsAgent exactly as before

Choosing the type for each job

Retail and travel sites that serve regional consumer pages call for residential proxies or residential premium proxies. High-volume calls to APIs and permissive sites are most economical on datacenter proxies. For carrier-specific data, use mobile proxies. All four types are listed on the proxies page, with rates on the pricing page; for bigger collection projects, our proxy for scraping overview goes further.

Questions team leads ask

How do we keep costs predictable? Usage is billed per gigabyte. Axios downloads only what each call requests, so most of the spend comes from request volume; cap concurrency and schedule jobs instead of polling continuously.

How do we know the service is still using the proxy? Log the exit IP at start-up with the ipify call above, and recheck the credentials in our proxy checker whenever something changes.

What are the rules for collection? Check each site's robots.txt and Terms of Service before a job goes live, keep request rates modest, slow down when a site answers 429, and limit collection to public data your company is allowed to use. Never point a proxy at authentication or anti-fraud controls to get past them, and never use one for fraud.

The key point to remember: use https-proxy-agent instead of Axios's native proxy field, so HTTPS traffic tunnels reliably. Everything else, from the shared client to retries, builds on that.