GIP blackout window (sequential)

What this is

In GIP, the term blackout window (sometimes “black window” in code or logs) is the per-store, per-connector–event record that says: “synchronous processing is only allowed for documents on or before this calendar line until earlier work catches up.”

When a connector–event mapping is marked sequential (isSequential: true), ingress may block new jobs if the business transaction date (by calendar day) is strictly after the stored blackout_date. Egress, after a terminal job result, updates that window (on failure) or clears it (on success) and can schedule deferred re-executions via a Mongo-backed retry queue and a small scheduler in the egress app.

Runtime code is split between apps/ingress-service, apps/egress-service, configuration via apps/app-connector-service (ACS), and shared Mongo + utilities in @libs/dip-v2.

When to use it

Enable isSequential on a connector–event mapping when:

• Events should be applied in business date order per store (or another key you read into storeId via sequentialKey), and
• A failed or stuck day should block or sequence later business days until the work succeeds or the window is cleared by egress.

If isSequential is false, the blackout and sequential-checkpoint paths described on this page do not run for that mapping.

Configuration you must set

Connector–event mapping (DipV2 + ACS)

ACS validation (CreateConnectorEventDto / ConnectorEventUpdateDto in apps/app-connector-service/src/dto/connector-event.ts): when isSequential is true, sequentialKey and transactionDatePath are required (@ValidateIf on those fields).

Outbound event (refDocPath)

The outbound event’s refDocPath (see Data model reference) drives refDocNo on the event log and job. That value participates in retry queue identity together with ssoEnterpriseId and connectorAppEventId (see below).

Egress environment

Date and identity helpers (@libs/dip-v2)

libs/dip-v2/src/utils/gip-blackout.ts defines the core comparisons:

• computeGipBlackoutDate(transactionDate) — Takes the calendar day of the transaction, adds one day, and returns YYYY-MM-DD (string). This is the synthetic “next line” used when persisting blackout_date on the window document and when updating / removing rows in GipBlackoutWindowRepository.
• isDocDateAfterBlackoutDate(transactionDate, blackoutDate) — true when the document’s calendar day is strictly after the blackout row’s calendar day (ignores time-of-day within the same calendar date).
• gipBlackoutEventName(ssoEnterpriseId, connectorAppEventId) — "{enterpriseId}-{mappingObjectId}" for retry queue event_name.
• gipRetryConsumerKey(ssoEnterpriseId, connectorAppEventId) — "{enterpriseId}_{mappingObjectId}" for consumer_name.

// libs/dip-v2/src/utils/gip-blackout.ts (conceptual)
export function computeGipBlackoutDate(transactionDate: Date): string {
  return moment(moment(transactionDate).format('YYYY-MM-DD'), 'YYYY-MM-DD')
    .add(1, 'days')
    .format('YYYY-MM-DD');
}

export function isDocDateAfterBlackoutDate(
  transactionDate: Date,
  blackoutDate: Date | string,
): boolean {
  const docDay = moment(transactionDate).format('YYYY-MM-DD');
  const blackoutDay = moment(blackoutDate).format('YYYY-MM-DD');
  return moment(docDay, 'YYYY-MM-DD').isAfter(moment(blackoutDay, 'YYYY-MM-DD'));
}

Mongo collections (DipV2 connection)

gip_outbound_sync_blackout_windows — blackout window

Model: GipBlackoutWindowModel. Unique index on (ssoEnterpriseId, connectorAppEventId, storeId).

gip_outbound_retry_queues (GIP outbound retry queue)

Model: GipBlackoutRetryQueueModel. Indexed on next_retry_at for sweeps. Important fields: event_name, doc_no, req (JSON with jobId for re-execution), event_rety_count, next_retry_at, consumer_name, blackout_window_details (e.g. blackout_source vs blackout_dependent).

gip_history_sync_blackout_window — history

When a window row is deleted after a successful removeDocno clears the last error docs, a snapshot is written to history before deletion (see GipBlackoutWindowRepository.removeDocno).

Ingress: job creation and first blackout check

File: apps/ingress-service/src/tasks/request-handler.ts.

1. For each connector–event mapping with isSequential true, read storeId from the payload with get(dto, connectorEventMapping.sequentialKey).
2. Resolve transactionDate from transactionDatePath (or now with a warning if the path is empty). If the path is set but missing/invalid, the job is BLOCKED with an explicit error (no blackout logic).
3. findBlackWindow for { ssoEnterpriseId, connectorAppEventId, storeId }.
4. If a window exists and isDocDateAfterBlackoutDate(transactionDate, blackWindow.blackout_date) is true:
   • Call saveBlackWindow with the same store/mapping/enterprise, refDocNo, and transactionDate.
   • Set job BLOCKED and error: 'GIP blackout dependent: waiting for prior sync window'.
5. Sequential checkpoint (same file): if not blocked by blackout, job sync checkpoint may still BLOCK the job for other sequential rules ('Job is blocked due to sequential issue') — that is ordering, not the same as blackout. See Egress runtime and retry for the overall job pipeline.
6. After jobs are created, for each job that is BLOCKED with the GIP blackout message, enqueue a row via gipBlackoutRetryQueueRepo.saveRetryQue, passing the current blackout document from another findBlackWindow (used to populate blackout_window_details in the queue repo layer).

Egress: after terminal job status

File: apps/egress-service/src/tasks/task-executor.ts — applyGipBlackoutAfterTerminalStatus.

Runs only if the mapping is isSequential and job.storeId is set.

Backoff (minutes) before the next next_retry_at: 4^(prev+1) + (10 * prev + 1) where prev is the prior event_rety_count (or 0). That interval is added to Date.now() in milliseconds.

// apps/egress-service/src/tasks/task-executor.ts (excerpt)
const maxRetries = Number(
  this.configService.get<string>('GIP_BLACKOUT_MAX_RETRIES') ?? '10',
);
const totalMin = Math.pow(4, prev + 1) + (10 * prev + 1);
const nextRetryAt = Date.now() + totalMin * 60 * 1000;

Egress: GipBlackoutRetryScheduler

File: apps/egress-service/src/scheduler/gip-blackout-retry.scheduler.ts.

• Fires on a fixed 5 minute interval (5 * 60 * 1000 ms).
• If IS_QUEUE_WORKER !== 'true', the handler returns immediately (no sweep).
• Loads retry rows with next_retry_at ≤ now, parses req JSON for jobId, and calls TaskExecutorService.executeJob(new ObjectId(jobId)).

End-to-end flow (mermaid)

flowchart TD
  Payload[Inbound JSON payload] --> RH[RequestHandler]
  RH --> Seq{isSequential?}
  Seq -->|no| JobPlain[Create job - no blackout path]
  Seq -->|yes| Store[Read storeId from sequentialKey]
  Store --> Tx[Resolve transactionDate from transactionDatePath]
  Tx --> FindBw[findBlackWindow]
  FindBw --> After{isDocDateAfterBlackoutDate?}
  After -->|yes| BlockJob[Job BLOCKED - GIP blackout dependent]
  After -->|no| Checkpoint[Job sync checkpoint PENDING or BLOCK other]
  BlockJob --> SaveQue[saveRetryQue with blackout ref]
  Checkpoint --> Dispatch{PENDING and async?}
  Dispatch -->|yes| Queue[Kafka dispatches job]
  Queue --> Eg[TaskExecutorService.executeJob]
  Bridge[Other job paths] --> Eg
  Eg --> Terminal[Terminal status]
  Terminal --> Bw[applyGipBlackoutAfterTerminalStatus]
  Bw -->|SUCCESS| Remove[removeDocno + deleteRetryQue]
  Bw -->|FAIL or filter failed| SaveBw[saveBlackWindow + saveRetryQue backoff]
  SaveBw --> Sch[GipBlackoutRetryScheduler 5m]
  Sch --> Eg

Operational notes

• Symptom: job error or log text starting with GIP blackout dependent — a document’s calendar day is after the current blackout_date for that store + mapping.
• Tuning: GIP_BLACKOUT_MAX_RETRIES; the sweep interval is 5 minutes in code unless changed in a future release.
• Where to configure mapping fields: operators use the G1 DIP / ACS surface backed by app-connector-service DTOs, not the ingress/egress apps directly.

Related

• Data model reference — transactionDatePath and mapping fields
• Ingress — async and sync — RequestHandler
• Egress runtime and retry — TaskExecutorService and Kafka
• ACS API overview — REST surface for connector–event mappings