Product Launch : abtestinghub

A/B Testing & Experimentation Frameworks

Role

Co-Founder & Lead Statistical Engineer

Industry

Growth Analytics & Experimentation

Duration

6 Months

Stage 1: Identifying the Problem & Market Research

Companies run A/B tests to improve conversion rates, but manual tracking & analysis slow down decision-making.
Growth teams needed an automated tool to track experiments, manage hypotheses, and run statistical analyses in real-time.
Goal: Build an integrated A/B testing dashboard to simplify experiment tracking & statistical evaluations.

Key Research Findings:

72% of growth teams struggle with experiment tracking & result interpretation.
Manual hypothesis management leads to inconsistencies in decision-making.

Stage 2: Designing the Experimentation Framework

Developed an end-to-end A/B testing system with built-in hypothesis tracking, experiment logs, and statistical calculators.
Created a power calculation tool to ensure experiments had enough data before launching.
Designed an interactive dashboard where users could track live experiments & analyze results instantly.

✅ Key Features:
✔️ Real-time Experiment Tracking – Live status updates on ongoing tests.
✔️ Hypothesis Management Tool – Stores & organizes test ideas & learnings.
✔️ Automated Statistical Analysis – Runs significance tests & MVT analysis.
✔️ Multi-Armed Bandit (MAB) Integration – Uses Reinforcement Learning for test optimization.

Stage 3: Development & Tech Stack

Built a scalable experimentation framework for real-time data collection & statistical computations.
Integrated Multi-Armed Bandit models (Batch Thompson Sampling) to dynamically allocate traffic to better-performing variants.
Designed a frontend dashboard with an easy-to-use UI for non-technical growth teams.

🛠️ Tech Stack:

Frontend: React.js, TailwindCSS
Backend: Django
Database: PostgreSQL
Statistical Engine: Python (Scipy, Statsmodels)
Machine Learning: Reinforcement Learning (MAB Models)

Stage 1: Identifying the Problem & Market Research

Companies run A/B tests to improve conversion rates, but manual tracking & analysis slow down decision-making.
Growth teams needed an automated tool to track experiments, manage hypotheses, and run statistical analyses in real-time.
Goal: Build an integrated A/B testing dashboard to simplify experiment tracking & statistical evaluations.

Key Research Findings:

72% of growth teams struggle with experiment tracking & result interpretation.
Manual hypothesis management leads to inconsistencies in decision-making.

Stage 2: Designing the Experimentation Framework

Developed an end-to-end A/B testing system with built-in hypothesis tracking, experiment logs, and statistical calculators.
Created a power calculation tool to ensure experiments had enough data before launching.
Designed an interactive dashboard where users could track live experiments & analyze results instantly.

Stage 3: Development & Tech Stack

Built a scalable experimentation framework for real-time data collection & statistical computations.
Integrated Multi-Armed Bandit models (Batch Thompson Sampling) to dynamically allocate traffic to better-performing variants.
Designed a frontend dashboard with an easy-to-use UI for non-technical growth teams.

🛠️ Tech Stack:

Frontend: React.js, TailwindCSS
Backend: Django
Database: PostgreSQL
Statistical Engine: Python (Scipy, Statsmodels)
Machine Learning: Reinforcement Learning (MAB Models)

Stage 4: Testing, Validation & User Feedback

Conducted beta testing with 5+ businesses running A/B tests.
Refined statistical accuracy, result interpretation, and UI experience based on feedback.
Implemented automated result summaries, making reports easier to digest.

📊 Impact:
✅ Cut A/B test analysis time by 60%.
✅ Increased experiment tracking efficiency by 3x.
✅ Enabled smarter decision-making for marketing & product teams.

Stage 5: Final Deployment & Adoption

Successfully scaled ABTestingHub into a structured experimentation management platform.
Developed an API integration for businesses to seamlessly incorporate real-time A/B test results into their workflows.
The platform became a go-to SaaS-based tool for structured, data-driven experimentation.
However, due to unpaid hosting fees, the website is currently down, but the platform remains a strong portfolio piece.

Reflections

Leading ABTestingHub as a Co-Founder was an incredible experience. It strengthened my ability to build and scale a product from scratch, integrate statistical rigor into real-world business experimentation, and create a platform that directly impacts data-driven decision-making.