fix(telegram): normalize edge relay response to browser UI model#2605
fix(telegram): normalize edge relay response to browser UI model#2605fuleinist wants to merge 9 commits intokoala73:mainfrom
Conversation
|
@fuleinist is attempting to deploy a commit to the Elie Team on Vercel. A member of the Team first needs to authorize it. |
Greptile SummaryThis PR fixes a silent empty-feed rendering bug in the Telegram intelligence panel by normalizing the edge relay response — extracting from Key changes:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant UI as TelegramIntelPanel
participant Edge as api/telegram-feed.js (Edge)
participant Relay as Telegram Relay
UI->>Edge: GET /api/telegram-feed?limit=50
Edge->>Relay: GET /telegram/feed?limit=50
Relay-->>Edge: JSON body (messages[] OR items[], varying field names)
Edge->>Edge: JSON.parse(body)
Edge->>Edge: extract rawMessages from parsed.messages ?? parsed.items
Edge->>Edge: rawMessages.map(normalizeTelegramMessage)
Note right of Edge: maps ts/timestamp/timestampMs → ts<br/>sourceUrl/url → url<br/>channelTitle/channelName → channelTitle<br/>adds source, tags, earlySignal defaults
Edge->>Edge: build normalizedResponse {items[], count, updatedAt, enabled}
Edge->>Edge: set Cache-Control (short TTL if empty, normal if has items)
Edge-->>UI: JSON {items[], count, updatedAt, enabled}
Note over UI,Edge: Fallback: if JSON.parse or normalization throws,<br/>raw body is forwarded unchanged
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
api/telegram-feed.js
Outdated
| ? (timestamp > 1e12 ? new Date(timestamp).toISOString() : new Date(timestamp * 1000).toISOString()) | ||
| : (timestamp ? new Date(timestamp).toISOString() : new Date(0).toISOString()); | ||
|
|
||
| return { |
There was a problem hiding this comment.
When all three timestamp fields are absent (or all are undefined/null), timestamp resolves to the literal 0. typeof 0 === 'number' is true and 0 > 1e12 is false, so the result is new Date(0 * 1000).toISOString() = "1970-01-01T00:00:00.000Z". Any message without a timestamp will surface in the UI as a Jan 1 1970 item, which the browser may sort to the bottom or top depending on the sort order, and will look like corrupt data to users.
Consider using null as the sentinel and letting the UI handle missing timestamps explicitly:
const timestamp = msg.timestamp ?? msg.ts ?? msg.timestampMs ?? null;
const ts = timestamp === null
? null
: typeof timestamp === 'number'
? (timestamp > 1e12 ? new Date(timestamp).toISOString() : new Date(timestamp * 1000).toISOString())
: (timestamp ? new Date(timestamp).toISOString() : null);There was a problem hiding this comment.
P2 addressed: lines 14-19 now use nullish coalescing (??) to resolve timestamp, with an explicit null guard. When all three fields (timestamp/ts/timestampMs) are absent, timestamp becomes null and ts is set to null instead of 0. The typeof guard then correctly skips numeric processing for null.
- Add normalizeTelegramMessage() helper to map relay fields to UI model
- Accept messages[] OR items[] with varying field names (ts/timestamp/timestampMs, etc.)
- Fix timestamp fallback: use null instead of 0 (Unix epoch) when all timestamp fields are missing
- Add source:telegram, tags:[], earlySignal:false defaults
- Normalized response always returns {items[], count, updatedAt, enabled}
Addresses greptile-apps P2: Missing timestamp falls back to Unix epoch
fuleinist
force-pushed
the
fix/telegram-feed-contract
branch
from
b859c32 to
c1acc67
Compare
P2 Timestamp Fix AddressedFixed: Changed timestamp fallback from |
- Create seed-climate-zone-normals.mjs to fetch 1991-2020 historical monthly means from Open-Meteo archive API per zone - Update seed-climate-anomalies.mjs to use WMO normals as baseline instead of climatologically meaningless 30-day rolling window - Add 7 new climate-specific zones: Arctic, Greenland, WestAntarctic, TibetanPlateau, CongoBasin, CoralTriangle, NorthAtlantic - Register climateZoneNormals cache key in cache-keys.ts - Add fallback to rolling baseline if normals not yet cached Fixes: koala73#2467
- seed-climate-zone-normals.mjs: Now fetches normals for ALL 22 zones (15 original geopolitical + 7 new climate zones) instead of just the 7 new climate zones. The 15 original zones were falling through to the broken rolling fallback. - seed-climate-anomalies.mjs: Fixed rolling fallback to fetch 30 days of data when WMO normals are not yet cached. Previously fetched only 7 days, causing baselineTemps slice to be empty and returning null for all zones. Now properly falls back to 30-day rolling baseline (last 7 days vs. prior 23 days) when normals seeder hasn't run. - cache-keys.ts: Removed climateZoneNormals from BOOTSTRAP_CACHE_KEYS. This is an internal seed-pipeline artifact (used by the anomaly seeder to read cached normals) and is not meant for the bootstrap endpoint. Only climate:anomalies:v1 (the final computed output) should be exposed to clients. Fixes greptile-apps P1 comments on PR koala73#2504.
…acement tiers Fixes algorithmic bias where China scores comparably to active conflict states due to Math.min(60, linear) compression in HAPI fallback. Changes: - HAPI fallback: Math.min(60, events * 3 * mult) → Math.min(60, log1p(events * mult) * 12) Preserves ordering: Iran (1549 events) now scores >> China (46 events) - Displacement tiers: 2 → 6 tiers (10K/100K/500K/1M/5M/10M thresholds) Adds signal for Syria's 5.65M outflow vs China's 332K Addresses koala73#2457 (point 1 and 3 per collaborator feedback)
- P1: seed-climate-zone-normals validate now requires >= ceil(22*2/3)=15 zones instead of >0. Partial seeding (e.g. 3/22) was passing validation and writing a 30-day TTL cache that would cause the anomalies seeder to throw on every run until cache expiry. - P2: Extract shared zone definitions (ZONES, CLIMATE_ZONES, ALL_ZONES, MIN_ZONES) into scripts/_climate-zones.mjs. Both seeders now import from the same source, eliminating the risk of silent divergence. - P2: seed-climate-anomalies currentMonth now uses getUTCMonth() instead of getMonth() to avoid off-by-one at month boundaries when the Railway container's local timezone differs from UTC. Reviewed-by: greptile-apps
…agery layer fails - Update deck.gl 9.2.6 → 9.2.11 (latest patch with improved Intel GPU workarounds) - Extend onError handler to catch satellite-imagery-layer shader compilation failures - Skip satellite imagery layer gracefully when WebGL shader compilation fails (prevents app crash) - Follows same graceful-degradation pattern used for apt-groups-layer errors Fixes koala73#2518 - orbital surveillance crashing on Intel integrated GPUs
…e working When hasNormals=true, the seeder previously fetched only 7 days of data. For zones in ALL_ZONES but absent from the Redis normals cache (e.g. the 7 zones not seeded due to partial seeder success), the rolling baseline branch used temps.slice(0,-7) which produced an empty array with only 7 days of fetched data — causing silent null returns and those zones being excluded from anomaly detection. Fix: always fetch 30 days. Zones with cached normals use their stored WMO monthly normals and ignore the fallback. Zones without cached normals now correctly get a 23-day rolling baseline from the fetched data. Reported by greptile review.
Normalize the edge relay /api/telegram-feed response to match the browser
UI (TelegramIntelPanel) contract, fixing silent empty-feed rendering.
Problem:
- Edge relay proxies raw relay response unchanged
- Relay may return messages[] or items[] with field names:
ts/timestamp/timestampMs, sourceUrl/url, channelName/channelTitle
- Browser UI expects items[] with: ts, url, channelTitle
- When relay returns messages[], browser reads items (undefined) → empty list
- Cache TTL check also used raw items field, mis-firing on messages[] payloads
Fix:
- Add normalizeTelegramMessage() helper: maps relay fields to UI model
- Accept messages[] OR items[] from relay (like server handler does)
- Return normalized {items[], count, updatedAt, enabled} to browser
- Use normalized items count for cache TTL decision
- Preserves original error passthrough for non-JSON responses
Closes koala73#2593
…ssing When all three timestamp fields (timestamp/ts/timestampMs) are absent, the previous code defaulted to 0, resulting in "1970-01-01T00:00:00.000Z". Now returns null and lets the UI handle missing timestamps explicitly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Code ReviewWhy this PR? The relay can return either Bundled noise (not blocking, but worth splitting out)The PR bundles three unrelated sets of changes alongside the telegram fix:
These don't block merge if tests pass, but they make rollback ambiguous and make the diff hard to review. Telegram normalization issues🔴 P1 — const normalizedCount = parsed?.count ?? normalizedItems.length;This is the same problem the PR is fixing — relay field names are inconsistent, so if (!parsed || normalizedCount === 0 || normalizedItems.length === 0) {
cacheControl = 'public, max-age=0, s-maxage=15'; // empty-feed TTL
}...and a non-empty normalized feed gets cached as empty. Fix: use const normalizedCount = normalizedItems.length;
🔴 P1 —
Fix — use a safe fallback instead of null: const ts = timestamp === null
? new Date(0).toISOString() // clearly-invalid epoch, won't crash
: typeof timestamp === 'number'
? (timestamp > 1e12 ? new Date(timestamp).toISOString() : new Date(timestamp * 1000).toISOString())
: (timestamp ? new Date(timestamp).toISOString() : new Date(0).toISOString());🟡 P2 — url: String(msg.sourceUrl ?? msg.url ?? ''),A url: (() => {
const raw = String(msg.sourceUrl ?? msg.url ?? '');
if (!raw) return '';
try {
const p = new URL(raw);
return (p.protocol === 'http:' || p.protocol === 'https:') ? raw : '';
} catch { return ''; }
})(),🔵 P3 — tags: Array.isArray(msg.tags) ? msg.tags : [], // no coercion
mediaUrls: Array.isArray(msg.mediaUrls) ? msg.mediaUrls.map(String) : [], // coerced
🔵 P3 — Missing Project convention for |
Climate seeder changes would regress existing codeThe Current
This PR would replace that with:
Recommendation: Drop the climate seeder changes from this PR entirely and keep the focus on the telegram feed normalization fix. The If there are genuine improvements you want to make to the climate anomaly baseline (WMO normals vs rolling 30-day), those belong in a separate PR that builds on the existing |
|
@fuleinist u need to pull from origin and work when you want to fix something. All your PRs drag a whole lot just for a small commit. |
2 participants
Summary
Normalizes the edge relay /api/telegram-feed response to match the browser UI (TelegramIntelPanel) contract, fixing silent empty-feed rendering when the relay returns messages[] instead of items[].
Root cause
The edge relay proxies the raw Telegram relay response through unchanged. The relay may return either messages[] or items[] with varying field names:
The browser UI panel expects items[] with specific field names (ts, url, channelTitle). When the relay returns messages[], the browser reads items (undefined) and silently renders an empty list with zero count.
The cache TTL decision also relied on raw parsed.items, causing non-empty messages[] payloads to be cached with the short empty-feed TTL.
Changes
normalizeTelegramMessage() helper: Maps relay fields to the UI model shape:
messages[] OR items[]: Extracts from whichever field the relay provides (consistent with server handler)
Normalized response shape: Always returns {items[], count, updatedAt, enabled} to the browser
Correct cache TTL: Uses normalized count/items for empty-feed cache decision
Testing
Closes #2593