Week Of: October 06, 2024
Most of this week was backend work to support enriching data and delivering better Dashboard features in-app.
🛠 New Features
- Hide Nonsensical Filters: When viewing narrative details, we now hide irrelevant filters like keyword, narrative, and threat score for cleaner results. Dive deeper into our Favorability MVP here.
🔧 Improvements
- Focused Cluster Query Configurations: Added configurations for the narratives API, including backfill dates and update frequencies for more tailored outputs.
- OS Writer for Generic Lists: Supports writing a generic enrichment under specific prefixes based on configuration. Check it out here.
🐞 Bug Fixes
- Tweet Truncation: Addressed an issue with ingested tweets having truncated text due to changes in the Twitter API or our pipeline.
🔍 Monitoring Chores
- FCQ Alerts: Implemented an alert system via Grafana to notify us if the FCQs don't run narrative detection in over 36 hours, ensuring our narrative insights stay fresh.
Full Changelog
Feature
- Write to new snowflake tables for smc_v8 & wmc_v6 that have enrichments field
- Focused Cluster Query additional configurations
- Hide AI summaries from BGOV version
- Create Vizzly Query Engine Layer
- Favorability - hide nonsensical filters when narrative details is shown
- Favorability - write the enrichments to OS
- favorability: coredata updated to write inference to common schema
- Favorability: os mapper updated to write fields to OS
- Vizzly PoC - Narratives details dash
- Update OS writer to support generic lists