Enercast — at the moment it matters
Enercast  /  How It Works  /  Guides

How to Scrape a Natural Gas Pipeline EBB

What pipeline bulletin boards post, how to collect it, and the problems that make it harder than it looks.

Updated November 18, 2026

Short answer: every U.S. interstate natural gas pipeline publishes its critical notices, operationally available capacity and scheduled quantities on a free public website called an Electronic Bulletin Board (EBB). You can collect them yourself with a headless browser and a parser per pipeline. It works, but there are 193 of them, they are built on different platforms, and they change without notice. This guide covers what is posted, how to collect it, and the problems you will hit.

What an EBB is and what it posts

FERC requires interstate pipelines that provide open-access transportation (18 CFR Part 284) to post transportation information on a public website, following the NAESB WGQ standards. The pages are usually labelled Informational Postings. Three parts matter to most people:

An EBB posts scheduled quantities, not metered flow. Scheduled volumes are usually close to what physically flows, but they are not the same thing, and they change from one nomination cycle to the next.

How to collect it

  1. Build your pipeline list. Start from the pipelines you care about and find each one’s Informational Postings page. It is usually linked from the operator’s website footer or customer section; FERC’s list of jurisdictional pipelines helps you check you have not missed one.
  2. Group pipelines by platform. Most EBBs run on a handful of shared platforms: Williams 1Line, Kinder Morgan DART, TC Energy’s TC eConnects, Enbridge LINK, Quorum-hosted sites and several others. One parser per platform covers many pipelines, but expect each pipeline to have its own quirks.
  3. Prefer the download over the page. Many capacity reports offer a CSV or Excel download. Where they do, it is more stable than parsing the HTML table.
  4. Use a headless browser where you must. Many reports are rendered with JavaScript or SQL Server Reporting Services (SSRS) and need a real browser (Playwright works well) to select the gas day and cycle before the data appears.
  5. Normalise everything to one schema. Pipeline, location ID, location name, direction, gas day, cycle, units, and each quantity column. Keep the raw text of every notice, because notices are revised.
  6. Schedule it politely. Space requests out (several seconds between pipelines), retry with back-off, and set timeouts. Respect each site’s terms of use.

The problems you will hit

The default cycle is not the latest cycle

Gas is nominated in cycles through the day (Timely, Evening, then the intraday cycles). Many SSRS-based reports open on the earliest cycle by default. A scraper that accepts the default will read the Timely cycle forever and look fine while being hours out of date. Select the cycle explicitly and check which cycle you got back.

“No data” does not always mean the pipeline is late

Some reports offer a settled “post” cycle that cannot exist until the gas day is over. Asking for it on the current day returns nothing, which looks like a pipeline posting a day late. Ask for the newest cycle that exists instead.

Units differ

Most interstate postings are in dekatherms (Dth); some sources you will compare against are in Mcf. The difference is about 3%, which is small enough to look like a real discrepancy.

The same gas is posted twice

Gas moving from one pipeline to another appears at the delivery point of the first and the receipt point of the second. Add up both and you have counted it twice. Decide which side of each interconnect you count.

Names collide

A loose name match will merge unrelated pipelines — matching “Transco” with a wildcard also picks up TransColorado. Key everything on pipeline and location ID, not on names.

Some sites block automated access

A few EBBs put reports behind a CAPTCHA or other bot protection. Treat that as the operator saying no: use what they publish elsewhere, or do without. Do not try to get around it.

Not all gas is on an EBB

Texas intrastate pipelines are regulated by the Railroad Commission, not FERC, and have no obligation to post capacity or scheduled quantities. Much of the gas that moves within Texas, including much of the gas exported to Mexico along the Gulf Coast, never appears on a public posting.

It never stays finished

Pipelines change platforms, rename locations, add columns and move reports. A collection that covers dozens of pipelines needs monitoring that tells you when a parser has gone quiet, not just when it errors.

A minimal example

For a capacity report that offers a CSV download, the core of a collector is short:

import io
import pandas as pd
import requests

# Copy the URL from the report's own download link; parameters vary by pipeline.
URL = "https://<pipeline-ebb>/capacity/download?gasDay=2026-11-18&cycle=ID1"

r = requests.get(URL, timeout=30, headers={"User-Agent": "your-name [email protected]"})
r.raise_for_status()
df = pd.read_csv(io.StringIO(r.text))
df.columns = df.columns.str.strip()
# Column names also vary by pipeline, e.g. Loc, Loc Name, Flow Ind, Sched Qty, Opr Cap

The work is everything around those lines: one of these per platform, the cycle selection, the browser automation for reports with no download, storage, revision tracking, monitoring, and keeping up when a site changes.

Or use the collected data

Enercast collects the EBBs of 193 interstate natural gas pipelines every 3 minutes through the trading day. Critical notices are scored 1–5 for market impact with an AI trading note; scheduled quantities are added up into daily supply and demand by category, with the meters behind every figure named. The website is open to everyone with no login, free alerts arrive with a one-hour delay, and Pro gets alerts within ~2 minutes of posting plus Excel/CSV downloads. Compare plans → · Other data sources →

Common questions

Is it legal to scrape a pipeline EBB?

EBB postings are public by FERC requirement, and collecting them is common. Each site has its own terms of use, so read them, keep your request rate low, and do not get around CAPTCHAs or other access controls. This is not legal advice.

Do pipeline EBBs show actual flows?

No. They post scheduled quantities by location and nomination cycle, plus operating and available capacity. Scheduled volumes are usually close to physical flow but are not metered measurements.

How many natural gas pipeline EBBs are there?

Enercast monitors 193 U.S. interstate natural gas pipelines. Most run on a small number of shared platforms, but each pipeline has its own report layout and quirks.

Is there an API for pipeline EBB data?

Pipelines generally do not offer a public API; the data is published as web reports and downloads. Enercast's Institutional plan provides critical-notice and supply and demand data by API.