Manas Singh | Data Engineer

</aside>

Gmail

</aside>

🎓 About

Data Engineer at Analytics Vidhya, building a centralized data warehouse on Databricks over multi-source data (PostgreSQL, MariaDB, MongoDB, GA4) on an S3-backed medallion architecture (Bronze → Silver → Gold) with Parquet/Snappy compression. Shipped Project Brahma, an archival pipeline that extracts MongoDB user-activity data, converts it to columnar Parquet, and stores it in Amazon S3 Glacier — cutting cold-data footprint by ~5x and meaningfully reducing storage cost. Currently developing Gold-layer aggregates that power downstream B2C and B2B reports.

Previously a Data Engineering Intern at Xebia, where I built a Spark-based batch pipeline for YouTube comment sentiment analysis and shipped Streamlit dashboards for non-technical stakeholders — the baseline that later evolved into a real-time, multi-engine streaming version using Apache Kafka, BERT, and Azure Text Analytics.

Graduated June 2025 with a Bachelor of Technology in Computer Science Engineering (Specialization in Big Data) from the University of Petroleum and Energy Studies (UPES), CGPA 7.95 / 10. Core stack: Python, Java, SQL (PostgreSQL, MySQL, MariaDB), Apache Spark / PySpark, Kafka, AWS (S3, S3 Glacier), Databricks (Delta Lake, Unity Catalog), Azure (Data Factory, Databricks), and BI tools (Power BI, Tableau, Streamlit).

Feel free to reach out at [email protected] for further information.

🚀 Experience

DATA ENGINEER | Analytics Vidhya | INDIA | Aug'25 - Present

Building the company's single source of truth — one central warehouse that pulls every database, app, and analytics feed (500+ GB of raw data) into one place that any team can query, instead of each team scraping its own copy. Stack: Databricks + S3, Parquet/Snappy compression (~5x size reduction, 500 GB → ~100 GB).
Designed how data flows from raw to reportable — a three-stage refinery (raw → cleaned → business-ready) so analysts pulling B2C and B2B reports don't have to redo the cleanup every time. Medallion architecture on Databricks (Bronze → Silver → Gold); Silver catalog live, Gold-layer aggregates in progress.
Built Project Brahma, the system that quietly moves old user-activity records off expensive hot storage into cheap deep storage — keeps history queryable when needed while cutting the cold-data bill substantially. Pipeline: MongoDB → JSON → Parquet (Snappy) → S3 Glacier, ~5x compression.
Worked across engineering, product, and analytics to agree on what each piece of data should look like before it enters the warehouse — so a schema change in the app doesn't quietly break a finance report two weeks later. Data contracts and ingestion schemas defined with the Spring Boot backend team.

DATA ENGINEERING INTERN | Xebia | REMOTE | Jun'24 - Aug'24

Built the engine that reads thousands of YouTube comments and tells you whether viewers actually loved a video or just clicked on it — useful for any creator or brand trying to figure out what's working. Spark batch pipeline over large-scale scraped comment datasets.
Designed the dashboards stakeholders actually used — point-and-click views of audience mood, engagement breakdowns, and per-video comparisons, no spreadsheets required. Streamlit.
This was the prototype that later grew into the real-time YouTube sentiment streaming system (see Projects) — the NLP classification baseline that made the live, multi-engine version possible.

👨🏽‍💻 Project Gallery

Projects

✨ Skills

Skills

⚡️ Companies

I had the privilege of working with

Jun’24 - Aug’24

Analytics Vidhya

Aug’25 - Present

⭐️ Achievements/Certifications

Microsoft Certified: Azure AI Engineer Associate — July 2024.
Java (Basic) Certified — HackerRank, May 2024.
Winner, Data Nerds Competition — Xebia TechFest, March 2023.
Built a real-time, multi-engine YouTube sentiment streaming system (Kafka + BERT + Azure Text Analytics) on top of the Xebia batch baseline.
Built an Olympics Data Analytics dashboard in Power BI on Azure Data Factory + Databricks pipelines.