Bulk operations

The Pivotal API doesn’t expose bulk endpoints (e.g. there is no POST /customers/bulk that takes an array). The reason: bulk endpoints either lie about partial-failure handling, or push the complexity into a job queue you can’t see into.

For now, fan out from your side. The patterns below handle 95% of bulk imports — backfills, migrations, periodic refreshes.

Sizing the work

Two numbers matter:

• Rate limit: 60 requests/min on a default key, 600/min on paid. Each bulk operation is N requests, where N is your record count.
• Worker concurrency: how many requests you have in flight at once.

A safe default for a 60/min key: concurrency of 4, with each task waiting briefly between requests. That keeps you well under the bucket while staying fast enough to finish 5,000 records in ~85 minutes.

For 600/min: concurrency of 10 finishes 5,000 records in under 10 minutes.

A bulk import script

import pLimit from "p-limit";
interface SourceCustomer { name: string; slug: string; domain: string; external_id: string }
const SOURCE: SourceCustomer[] = await loadFromCsv("./customers.csv");
const CHECKPOINT_FILE = "./bulk-import.checkpoint";
// Resume support: skip records we've already pushed.
const done = new Set<string>(
  (await Bun.file(CHECKPOINT_FILE).text().catch(() => "")).split("\n").filter(Boolean),
);
const limit = pLimit(4); // concurrency
let errors = 0;
async function importOne(c: SourceCustomer) {
  if (done.has(c.external_id)) return;
  const res = await fetch("https://my.pivotal.app/api/v1/customers", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.PIVOTAL_API_KEY}`,
      "Content-Type": "application/json",
      "Idempotency-Key": `bulk-import-${c.external_id}`,
    },
    body: JSON.stringify({
      name: c.name,
      slug: c.slug,
      domain: c.domain,
      status: "onboarding",
      metadata: { external_id: c.external_id },
    }),
  });
  if (res.status === 429) {
    const wait = Number(res.headers.get("Retry-After") ?? 5);
    await new Promise((r) => setTimeout(r, wait * 1000));
    return importOne(c); // retry
  }
  const body = await res.json();
  if (!res.ok && body.error?.code !== "slug_taken") {
    errors += 1;
    console.error(`✗ ${c.external_id}:`, body.error?.code, body.error?.message);
    return;
  }
  done.add(c.external_id);
  await Bun.write(CHECKPOINT_FILE, [...done].join("\n"));
  console.log(`✓ ${c.external_id} → ${body.id}`);
}
await Promise.all(SOURCE.map((c) => limit(() => importOne(c))));
console.log(`Done. ${done.size} succeeded, ${errors} failed.`);

Three things to call out:

1. Idempotency-Key lets you re-run after a crash without duplicating records.
2. The checkpoint file is a flat-text list of external_ids. Crude but enough for a one-shot script. For long-running services, use Redis or a Postgres row.
3. slug_taken is accepted as a non-error — it means the customer was created in a previous run.

Why per-record Idempotency-Key

If you send the same body to POST /customers twice without a key, you get 409 slug_taken on the second attempt. With a key, you get 200 OK with the original response — which lets the script keep moving and treat it as “already done”.

Throttling against X-RateLimit-Remaining

The script above retries on 429. You can do better — slow down before you hit zero:

async function importOne(c: SourceCustomer) {
  const res = await fetch(/* ... */);
  const remaining = Number(res.headers.get("X-RateLimit-Remaining") ?? 100);
  if (remaining < 5) await new Promise((r) => setTimeout(r, 1000));
  // ... rest of the handler
}

This drops your peak throughput slightly but eliminates 429 round trips entirely.

What to do about contacts and onboardings

Bulk-importing customers is the main case. If you also need to attach contacts and onboardings, fan them out the same way after the customer pass finishes:

// Pass 1: customers
await Promise.all(SOURCE.map((c) => limit(() => importCustomer(c))));
// Pass 2: contacts (now we have Pivotal customer ids)
const idMap = await loadCheckpoint("./bulk-import.checkpoint.contacts");
await Promise.all(SOURCE.flatMap((c) =>
  c.contacts.map((contact) => limit(() => importContact(c.external_id, contact))),
));
// Pass 3: onboardings (last — they read back the customer state)
await Promise.all(SOURCE.map((c) => limit(() => importOnboarding(c))));

Three passes is more wall time but easier to reason about than interleaving. The total request count is the same.

Reporting

After the script finishes, eyeball:

• Checkpoint count vs source count — should match.
• Errors — anything that isn’t slug_taken, look at the error and decide.
• The Pivotal API Keys page — confirms the request count tracks what your script logged.

What NOT to do

• Don’t disable retries. Network blips are real. The script above retries automatically on 429; add retries on transient 5xx too.
• Don’t share keys across bulk jobs. One key per bulk run makes the API Keys page a useful audit trail.
• Don’t fan out without a concurrency limit. Promise.all over 5,000 records means 5,000 in-flight requests. You’ll trip the rate limit and the OS file descriptor limit before you trip anything else.
• Don’t import then mutate the same row in the same pass. Two requests against the same record concurrently is a race. Let pass 1 finish before mutating.