Technology Associate Intern @ZS Associates | Ex Backend @ VarsityPro

HIMANSHU SHARMA • Peerlist

Built an end-to-end data pipeline leveraging Docker, Airflow, DBT, and SQL databases to automate data ingestion, transformation, and storage. The pipeline ensures scalability, consistency, and persistence, with containerized services and modular data transformations for real-world applications.

Automated workflows using Apache Airflow to orchestrate tasks and Kafka for real-time data streaming. Integrated cron jobs and Docker volumes for seamless scheduling and data persistence, ensuring timely and reliable updates for analytics and reporting.

ELT Pipeline

Developed a question answering bot using Falcon LLM open source model.

Falcon LLM based Question-Answer Bot

• Galaxy Search is a search engine inspired by NASA's APOD website, designed to provide easy access to astronomy pictures. It utilizes ElasticSearch for efficient data indexing and retrieval, with FastAPI managing the backend and Streamlit offering an interactive frontend interface. 
• It allows seamless document insertion and search functionality. ElasticSearch is containerized using Docker.

Galaxy Search

This project utilizes the GitHub Firehose, a live feed of JSON events, to create a real-time data pipeline
By integrating Python, Kafka, and the Quick Streams library, it ensures efficient ingestion, processing, and streaming of GitHub events to enable actionable insights into open-source activities.

Github Data Stream

• Handlo is an advanced platform for image context retrieval and caption generation, utilizing models like BLIP and TinyLlama for natural language processing and computer vision tasks. With a modular architecture, it includes features like hashtag suggestion, vector-based storage with Qdrant, and a Retrieval-Augmented Generation (RAG) approach for efficient and scalable content processing.
• Handlo's HashtagRetriever leverages Qdrant vector databases and LlamaIndex to generate hashtags based on input sentences. It uses embeddings, Hugging Face APIs, and context-specific search to suggest mood, object, and place-related hashtags. Modular and scalable, it ensures efficient retrieval of relevant hashtags, even handling cases with insufficient sentence information gracefully.

Handlo

Designed and implemented a product recognition solution for identifying, classifying various products McCain Brand.

McCain Product Recognition

Live website: https://openocr-ef1r.onrender.com
Deployed a Website made in streamlit performs OCR using Google Cloud Vision API , performs Realtime OCR also provide feature to change formats of image among tiff, jpeg, png.

OpenOCR

• PeerRAG is a RAG chatbot integrating LangChain, FAISS, and HuggingFaceEmbeddings for context-aware query
handling using advanced NLP and vector search.
• Engineered a comprehensive employee dataset by scraping Peerlist using BeautifulSoup and FireCrawl; enhanced chatbot
intelligence, enabling data retrieval for over 500 distinct employee profiles, improving response accuracy by 40%

PeerRAG

• This project leverages Python, Selenium, and Django to automate web scraping, focusing on Product Scraping on scheduled interval. Data is then parsed using Beautiful Soup and stored in a Django database for ongoing analysis and tracking.
• The project utilizes Celery for task scheduling, enabling asynchronous scraping operations. By combining Selenium's browser automation with Django’s ORM capabilities, it efficiently collects and manages data, tracking trends over time. The integration of Redis for task queue management ensures smooth execution, while dynamic data handling techniques allow the scraper to adapt to changing website structures and content.

Scheduled Product Scraper

This project is about analyzing different terrains through convolutional neural network. Classified the three terrains using the satellite imagery(LANDSAT-8 satellite) from January 2014 to March 2022
of 3 regions of India having Desert, Mountain, Plain terrain