Idempotency & retries¶
The same money operation can reach your wallet more than once. This page explains when and why, what timeouts and retries 19flip uses, and exactly how your wallet must deduplicate.
Rule: one idempotency_key = at most one money movement, and every repeat gets the same answer.
Why you will see repeats¶
| Source | When | Same idempotency_key? |
Same body? |
|---|---|---|---|
| 19flip transport retry | Your endpoint answered 5xx, timed out, or the connection failed |
yes | yes, byte for byte |
| Game repeat | A game did not get a definite answer and asks again to learn the outcome (for example Bingo Bongo retries unpaid wins with growing delays until you confirm, possibly for hours) | yes | yes |
| Manual retry by 19flip staff | An operation with an unknown outcome is re-sent from the 19flip admin panel (wins and rollbacks only) | yes | same fields; round_id may be a string of digits, game_id may now be filled |
| Manual bet rollback by 19flip staff | A bet with an unknown outcome is reversed | new key manual-rb-<number>; original_transaction_id points at the bet |
– |
A transport retry can arrive while your first attempt is still running: when we time out after 10 seconds we do not know whether you finished, and your server keeps processing the first request. Your deduplication must be safe under concurrency, not only sequentially.
Timeouts and retries (19flip → your wallet)¶
| Parameter | Value |
|---|---|
| Timeout per attempt | 10 s (whole request: connect, send, receive) |
| Attempts | 3 in total (the first try + up to 2 retries) |
| Pause before retry | 1 s before the 2nd attempt, 2 s before the 3rd (1 s × 2^n) |
| Retried on | HTTP 5xx, timeout, connection error (DNS, refused, reset, TLS) |
| Not retried | HTTP 4xx (a definitive answer), and any other non-2xx status such as a redirect |
| Worst case until the game gets an answer | about 33 s (10 + 1 + 10 + 2 + 10) |
| Connection test | 5 s timeout, 1 attempt |
These are the current defaults for all casinos. 19flip can tune them; any change will be announced to you in advance.
/round/finish (Flip Jet, moves no money) uses the same 3 attempts. If they all fail with 5xx, a timeout or a connection error, 19flip tries again later: after 1, 5, 15 and 60 minutes, then after 3, 6 and 12 hours (8 deliveries in total), each with the same body and idempotency_key. A 4xx answer ends the retries. Rounds whose callback was lost can be fetched from the round history.
After the last failed attempt the game receives an error and the operation is recorded on our side as outcome unknown. It is then settled in one of these ways, all of which rely on your deduplication:
- The game repeats the same operation with the same key (and you return the stored answer, or apply it if you never saw it).
- The game or 19flip staff rolls the bet back with
/bet/rollback, referencing it by itsidempotency_key(because we never received yourtransaction_id). - 19flip staff resolve it manually with you. For that, you must be able to look up an operation by
idempotency_key.
What your wallet must do¶
- Deduplicate on
idempotency_keyalone, across all players, sessions, games and endpoints. Put a UNIQUE constraint on it. Do not combine it withplayer_idorround_id. - Store the answer you gave (HTTP status and body) together with the key, in the same database transaction as the balance change.
- Return the stored answer for a repeat: same HTTP status, same
transaction_id, samebalance_before, samebalance_after, samestatus. Do not return the current balance, and do not move money again. - Repeat declines too. If you answered
402to a key, answer402to its repeats. Do not re-evaluate the balance under the same key. - Serialize concurrent requests with the same key. The second request must wait for the first and then return its stored answer, or fail with
5xxso that it is retried. It must never apply the operation a second time. - If a repeat has a different body (different amount or player) under a known key, do not apply it. Return the stored answer and alert your team: it indicates a bug, not a legitimate operation.
- Keep keys for a long time. Wins can arrive hours after the round and manual resolution can happen days later. Keep keys at least as long as you keep transaction records, and never less than 30 days.
- Rollbacks are at most once per original. A second rollback of the same original (under a different key) returns the first rollback's result and moves no money.
Reference implementation (debit)¶
Python with PostgreSQL (asyncpg). The same pattern applies to /win/credit and the rollbacks.
import json
import uuid
from decimal import Decimal
# CREATE TABLE wallet_ops (
# idempotency_key text PRIMARY KEY,
# kind text NOT NULL, -- BET_DEBIT, WIN_CREDIT, ...
# player_id text NOT NULL,
# amount numeric(18,2) NOT NULL,
# http_status int NOT NULL,
# response jsonb NOT NULL,
# transaction_id text UNIQUE, -- your id, returned to 19flip
# created_at timestamptz NOT NULL DEFAULT now()
# );
async def bet_debit(conn, body: dict) -> tuple[int, dict]:
key = body["idempotency_key"]
async with conn.transaction():
# Serialize concurrent requests with the same key.
await conn.execute("SELECT pg_advisory_xact_lock(hashtext($1))", key)
stored = await conn.fetchrow(
"SELECT http_status, response FROM wallet_ops WHERE idempotency_key = $1", key)
if stored:
return stored["http_status"], json.loads(stored["response"]) # identical answer, no money moved
player = await conn.fetchrow(
"SELECT balance FROM players WHERE id = $1 FOR UPDATE", body["player_id"])
if player is None:
return 404, {"error": "player not found"}
amount = Decimal(str(body["amount"]))
before = player["balance"]
if before < amount:
status, response = 402, {"error": "insufficient balance", "balance": float(before)}
else:
after = before - amount
await conn.execute("UPDATE players SET balance = $1 WHERE id = $2", after, body["player_id"])
tx_id = str(uuid.uuid4())
status, response = 200, {
"transaction_id": tx_id,
"balance_before": float(before),
"balance_after": float(after),
"status": "SUCCESS",
}
await conn.execute(
"INSERT INTO wallet_ops (idempotency_key, kind, player_id, amount, http_status, response, transaction_id)"
" VALUES ($1, 'BET_DEBIT', $2, $3, $4, $5, $6)",
key, body["player_id"], amount, status, json.dumps(response), response.get("transaction_id"))
return status, response
Notes:
- The
404for an unknown player is not stored, so a repeat is evaluated again. Store it too if you prefer. - Money is
Decimal/numeric, never float arithmetic. Convert to a JSON number only in the response.
What 19flip does on its side¶
19flip deduplicates too, which reduces but does not remove repeats to your wallet:
- An operation whose outcome is known (you answered
2xx, or a definitive4xx) is stored under its key. When the game repeats it, 19flip returns the stored result without calling you. - An operation whose outcome is unknown (
5xx, timeout, connection error after all attempts) is sent to you again when the game repeats it with the same key. - Transport retries happen inside a single call and are not deduplicated on our side – that is your job.
- 19flip treats
idempotency_keyas unique across all games, so you can too.