VibeCoach: A Camera-Free, TinyML Wearable That Corrects Workout Form in Real Time

Table of Contents

1. Key Highlights
2. Introduction
3. Why exercise form matters — and why it's hard to fix
4. How the prototype senses motion: hardware choices and why they matter
5. Turning raw sensor output into reliable orientation: Madgwick filter and calibration
6. The TinyML model: dataset strategy, features, and performance
7. Deploying the model on the MCU: CMSIS-Pack and firmware integration
8. Real-time feedback logic: avoiding noise-driven false alerts
9. Firmware specifics that enable reliable sampling and low latency
10. Power management and safety considerations for a wearable node
11. Validation, limitations, and what the prototype proves
12. Real-world scenarios and examples of use
13. Path to production: what would it take
14. Privacy and ethical implications
15. Practical advice for teams building similar systems
16. What’s next for VibeCoach — roadmap items and research directions
17. Final perspective on the prototype’s significance
18. FAQ

Key Highlights

• VibeCoach embeds a TinyML movement classifier and haptic feedback in a standalone wearable to detect and correct exercise form without cameras or internet connectivity.
• Prototype focuses on lateral raises using an MPU-9250 IMU and STM32F411 MCU, achieving 90.1% validation accuracy and delivering graded vibration alerts tied to error severity.
• Design prioritizes privacy, low latency, and battery life through on-device inference, Madgwick sensor fusion, non-blocking firmware, and careful power-management strategies.

Introduction

Many gym-goers push hard and do not see the results they expect because small deviations in movement shift load away from target muscles or create injury risk. VibeCoach confronts that problem with a different approach: rather than recording video or streaming sensor data to the cloud, it runs a lightweight neural network directly on an embedded microcontroller and tells the user—through vibration and LED cues—when their form goes wrong. The prototype targets the lateral raise, a deceptively simple exercise that requires precise shoulder orientation for effective deltoid activation. By combining precise orientation filtering, targeted calibration, and an Edge Impulse-trained TinyML model, VibeCoach demonstrates cameraless, private, and immediate form guidance suitable for public gym use.

This article explains the engineering choices behind VibeCoach, how the system works end-to-end, what the prototype proves, and what remains to be done to move from proof of concept to a production wearable.

Why exercise form matters — and why it's hard to fix

A few degrees of misalignment can change which muscles bear the load during an exercise. For isolation movements like the lateral raise, even 10–15° of deviation can push tens of percent of the mechanical load from the target deltoid to compensating muscles such as the upper trapezius. Repeatedly training with such misalignment reduces efficiency and increases injury risk—from rotator cuff strain to spinal issues under heavier loads.

Two structural obstacles prevent most lifters from fixing form consistently:

• Access to expert coaching is limited. Surveys and industry data indicate a large majority of gym members train without a personal trainer, often because of cost.
• Camera-based feedback is intrusive. Recording in a public gym is awkward, may disrupt others, and raises privacy concerns—especially for some groups who prefer not to be filmed.

VibeCoach addresses both: a compact, strap-mounted node monitors movement with a 9-axis IMU and gives immediate haptic corrections without capturing images or requiring a smartphone.

How the prototype senses motion: hardware choices and why they matter

The VibeCoach prototype balances measurement fidelity, compute capability, and wearability. Each component was chosen to support the mission of offline, real-time classification.

• STM32F411CEU6 (Black Pill) — core processing unit
  • ARM Cortex-M4 core with an FPU running at a tuned 50 MHz system clock. The FPU accelerates floating-point math used in sensor fusion and TinyML inference while balancing battery life. The MCU offers 512 KB flash and 128 KB SRAM, enough to host the model runtime and peripheral drivers.
• MPU-9250 9-axis IMU — precision motion tracking
  • The magnetometer in the MPU-9250 reduces orientation drift that would otherwise accumulate with accelerometer/gyroscope-only IMUs. This becomes important when monitoring subtle posture deviations across repeated reps.
• ST7735 1.8-inch LCD + 1×4 membrane keypad — on-device interface
  • A compact display and keypad allow users to choose exercise modes and calibrate without a phone. The physical UI supports the project’s offline, standalone ethos.
• RGB LED + mini vibration motor — multimodal feedback
  • Visual feedback follows a simple traffic-light metaphor (green = correct, yellow/blue = caution, red = critical). A vibration motor provides private, immediate correction cues that users can feel without interrupting concentration.
• 250 mAh Li-Po battery + TP4056 USB-C charger — compact power
  • The battery selection emphasizes a small physical footprint while providing sufficient operational time for prototyping sessions. USB-C charging via TP4056 supports widespread chargers.

Each choice supported an overriding constraint: keep the device small and portable while ensuring the sensor pipeline and classifier run reliably on-device.

Turning raw sensor output into reliable orientation: Madgwick filter and calibration

Raw accelerometer and gyroscope outputs are noisy and subject to drift. Converting those signals into stable orientation requires sensor fusion. VibeCoach uses the Madgwick filter for two practical reasons: low computational footprint and solid accuracy at modest sampling rates.

Why Madgwick?

• Computational efficiency: The Madgwick algorithm avoids expensive matrix algebra associated with some Kalman filter implementations. That matters on a Cortex-M4 running at 50 MHz, where cycles and RAM are limited.
• Robust at low sampling frequencies: Madgwick delivers stable quaternion outputs even when sampling is not extremely fast, which helps conserve battery.

Calibration to a user-specific baseline Rather than assuming a universal "zero" pose, VibeCoach includes a reference-pose calibration. During a short countdown the device collects ~300 samples and computes an average baseline quaternion (q_base). Subsequent orientation samples are multiplied by the inverse of that baseline (invQ), transforming the data into a normalized coordinate frame where the user's starting posture represents 0°. Normalizing in this way reduces inter-user variability and improves the classifier's ability to focus on deviations that matter for the exercise.

This pipeline—raw IMU → Madgwick filter → invQ calibration—creates stable, discriminative features for the TinyML model.

The TinyML model: dataset strategy, features, and performance

Training an on-device classifier required carefully shaped data and feature engineering to run within the MCU’s constraints.

Data collection

• Movement samples were captured through the Edge Impulse data-forwarder from the MCU connected via a USB-to-UART module. A mix of correct-form examples and intentional incorrect forms collected from an experienced lifter formed the initial dataset.
• The initial dataset targeted lateral raises exclusively as a focused proof of concept for deltoid form detection. Narrow scope helped sharpen model performance before scaling to more exercises.

Feature extraction and impulse design

• Window size: 1,000 ms (1 second) with a 100 ms stride. A one-second window captures a full repetition phase for many gym movements while overlapping windows ensure near-instantaneous inference.
• Spectral analysis: Quaternion time series were processed with FFT-derived spectral features to capture movement periodicity and power—effective at separating controlled reps from erratic motion caused by fatigue or poor technique.
• Classification block: A Keras neural network mapped processed windows into three classes: idle, correct_move, wrong_move.

Training results

• The trained model reached a validation accuracy of 90.1% with a loss of 0.27 in Edge Impulse Studio. Class-level metrics showed:
  • IDLE: 96.7% accuracy and 0.97 F1 score—important to avoid false positives while resting.
  • CORRECT_MOVE: ~91.6% accuracy.
  • WRONG_MOVE: lower separation, with about 18.2% of wrong_move windows misclassified as correct_move. This overlap highlights the need for more diverse incorrect-form data.

Practical meaning A 90% model at prototype scale demonstrates that the signal pipeline—from Madgwick-filtered quaternions through invQ normalization to spectral features—creates robust input for classification. The remaining classification overlap can be reduced by expanding the dataset to include more subjects, varying body sizes, and additional error patterns.

Deploying the model on the MCU: CMSIS-Pack and firmware integration

Model training is only half the story. The other half is making the inference run reliably on the STM32 without external dependencies.

Export path and integration

• Edge Impulse exports the trained model as a CMSIS-Pack tailored to ARM Cortex-M devices. Importing this pack into STM32CubeMX simplifies include paths and hardware-specific configuration.
• The generated code is merged into the CubeMX/STM32CubeIDE project. Because Edge Impulse’s runtime uses C++ classes, the main application migrated to main.cpp and wrapped necessary C headers in extern "C" blocks to preserve compatibility with the ST HAL drivers.

Why CMSIS-Pack matters

• The CMSIS-Pack bundles neural network architecture, spectral feature extraction blocks, and math optimizations tuned for ARM cores. This avoids manual porting errors and keeps the build toolchain consistent.

Real-time constraints

• The TinyML inference is invoked from the main loop where calibrated quaternion data are pushed into the classifier buffer. Ensuring that this loop runs without jitter required careful peripheral configuration and a non-blocking firmware model.

Real-time feedback logic: avoiding noise-driven false alerts

Immediate feedback is only useful when it is reliable. VibeCoach’s firmware prevents spurious vibrations via a graded scoring system and non-blocking state machine.

Score accumulator logic

• Each inference window classified as wrong_move with confidence >= 70% adds +2 to a wrongness accumulator score.
• Correct_move or idle windows decrement the score by -1 down to a floor of zero.
• This avoids triggering a correction on a single noisy frame.

Graduated haptic responses

• Score 4–7 (Information): Single short pulse at low duty cycle.
• Score 8–11 (Warning): Two medium pulses with a 1.5s cooldown.
• Score >= 12 (Strong correction): Three intense bursts with a 3s cooldown to prevent alarm fatigue or disruptive continuous vibration.

Non-blocking design

• Timers and HAL_GetTick() are used to schedule pulse durations and cooldowns without blocking the MCU. That preserves the continuous 10 ms sampling cadence and ensures inference windows are not skipped due to blocking delays.

Visual verification

• The RGB LED provides a secondary, coach-friendly cue. Green indicates acceptable form; blue/yellow indicates caution; red indicates critical deviation.

Real-world effect

• This multimodal approach means the device only vibrates when a pattern of faulty repetitions is detected—reducing false corrections and making haptic alerts actionable.

Firmware specifics that enable reliable sampling and low latency

Every millisecond counts when working with real-time motion data and constrained hardware.

Peripheral tuning

• I2C1 is configured in Fast Mode at 400 kHz to poll the MPU-9250 quickly and avoid sensor bottlenecks. Fast I2C prevents data lagging that would otherwise break the sequential windows required by the classifier.
• SPI1 is set for high-speed one-way transmission to the ST7735 LCD, ensuring smooth UI updates at ~6.25 Mbit/s.
• GPIOs for input (keypad) use internal pull-ups to keep the PCB minimal; RGB LED outputs run in push-pull mode for crisp visual state changes.

Timers and PWM

• TIM1 drives the vibration motor via PWM with tunable duty cycles for graded intensity.
• TIM2 and TIM3 drive the RGB LED channels to achieve precise color mixing.

Clock configuration and power trade-offs

• The MCU runs at an intentionally tuned 50 MHz rather than maximum rated frequency. This selection provides adequate compute headroom for Madgwick filtering and inference while extending battery life on the small Li-Po pack.

Serial for development

• USART remains enabled for data-forwarding and debugging during early stages, enabling in-field dataset capture and iterative model refinement.

Power management and safety considerations for a wearable node

Using a small Li-Po battery requires careful analog and software strategies to ensure safety and smooth operation.

Battery measurement

• A simple voltage divider using matched resistors scales the battery voltage into the MCU ADC range. Software then recovers the true battery voltage.
• ADC sampling is expensive if naively implemented. The firmware pre-warms a cached ADC result at boot and uses a non-blocking, averaged sampling pipeline (batch average of 32 samples passed through an EMA filter) to deliver smooth battery estimates without blocking the main loop.

Hysteresis and UX stability

• Software hysteresis prevents the battery indicator from jittering on-screen during transient current draws (e.g., when the vibration motor engages). The display only updates when the change exceeds a small voltage threshold or a percent gate.

Charging and charging safety

• TP4056 provides a standard linear charge management path via USB-C. Real productization would require more robust battery management (thermal protection, charge current negotiation) and regulatory validation, but the prototyping approach aligns with quick iterations.

Wearable safety

• Vibration intensity and cooldown periods are calibrated to avoid startling the user or causing discomfort during heavy lifts. The device is strap-mounted with modular placement so it can be positioned where haptic cues are most perceivable.

Validation, limitations, and what the prototype proves

Validation results

• The TinyML model yielded a 90.1% validation accuracy in early experiments. IDLE detection is especially robust (96.7%), reducing false alarms during rest between sets.

What the prototype proves

• TinyML inference for movement classification can run reliably on a low-power MCU with careful signal processing, sensor fusion, and firmware design.
• Camera-free monitoring preserves privacy and is more comfortable to use in public spaces.
• Haptic feedback can be effective for immediate correction when married to a score-accumulator to reduce noise-driven triggers.

Current limitations

• Dataset size and diversity: Training data came from a smaller sample and prioritized a single experienced subject. The model underperforms when presented with untrained incorrect patterns or atypical body morphologies.
• Single-exercise focus: The current pipeline is tuned for lateral raises. Adapting to different movements will require additional data and possible feature adjustments.
• No BLE connectivity yet: The device is offline by design. Adding wireless telemetry for logging and coaching apps is planned but not present in the prototyping phase.
• Prototyping form factor: The build uses development boards and off-the-shelf modules rather than a custom SMD PCB, making it larger than a production wearable would be.

Real-world scenarios and examples of use

Example 1: An intermediate lifter struggling with deltoid development

• Problem: The lifter performs lateral raises but feels fatigue in the traps and upper back rather than the side delts.
• VibeCoach placement: Strap the node to the upper arm.
• Outcome: The device calibrates baseline and monitors arm orientation. When scapular elevation or shoulder-abduction angle drifts, the unit issues a medium haptic pulse, prompting micro-adjustment. Over several sessions the lifter learns to maintain proper abduction angle and to avoid trap-dominant substitutions.

Example 2: Busy gym with no trainer and privacy concerns

• Problem: A user wants feedback but does not want to set up a camera or draw attention.
• VibeCoach advantage: No camera, no smartphone required. The haptic cues are private and the LED is discreet. The user receives corrections without recording video or broadcasting data.

Example 3: Coach-assisted group training

• Problem: A coach wants quick signals when athletes deviate from form during circuits.
• VibeCoach as augmentation: Wearables provide private cues to athletes, enabling coaches to spot-check or verify via the LED. Later, BLE (planned) can stream metadata for deeper review.

Real-world comparison to camera systems

• Cameras can deliver detailed joint-angle tracking but introduce privacy barriers, setup friction, occlusion errors, and latency when cloud-based inference is used.
• IMU-based approaches suffer from limited absolute spatial context but excel at wearable convenience, privacy, and continuous real-time feedback without line-of-sight constraints.

Path to production: what would it take

To evolve VibeCoach from prototype to product requires coordinated work across multiple domains.

Dataset expansion and model generalization

• Recruit diverse subjects across sex, age, body types, and lifting styles to gather both correct and incorrect movement variants. More negative examples sharpen decision boundaries and reduce misclassification of wrong_move as correct_move.

Hardware miniaturization

• Move from dev modules to an SMD PCB with a microcontroller, IMU, power management, vibrator driver, and a small connector or integrated battery. Optimized antenna placement and shielding will be necessary if BLE is added.

Regulatory and safety testing

• Battery safety testing, electromagnetic compatibility (EMC), and wearable-device regulations. For markets such as the EU, CE marking and for the U.S., FCC/UL considerations will be necessary depending on radio integration.

User experience design

• Form factor ergonomics (silicone strap, breathable materials), haptic intensity tuning, and an intuitive UI flow for calibration and exercise selection.
• Companion app (optional): BLE support for session logging, long-term analytics, and coach integration.

Business model considerations

• Bundled hardware plus optional subscription for advanced analytics and cloud coaching is one path. Alternatively, selling the hardware with free local-only features and paid cloud features for coaches or gyms is another.

Manufacturing and cost constraints

• Component downselection (e.g., choosing an MCU with integrated BLE vs discrete modules) will affect BOM costs and development complexity. Design for testability and assembly is critical.

Privacy and ethical implications

VibeCoach’s camera-free design addresses two ethical dimensions: user privacy and data minimization.

• No images or audio are captured, minimizing sensitive data collection and easing privacy concerns for users exercising in shared spaces.
• On-device inference means sensitive movement data need not leave the wearable unless the user opts in (future BLE/cloud features).
• Transparency is crucial. Any product should disclose what signals are collected, how long they are stored, and how users can delete or export their data.

Ethical considerations for feedback

• Haptic feedback should be unobtrusive and calibrated to avoid startling or distracting during heavy lifts.
• The device must avoid delivering misleading corrections. Conservatively tuned thresholds and coach validation can mitigate over-reliance on automated cues.

Practical advice for teams building similar systems

• Start narrow: Focus on a single exercise or a small set of exercises to validate the core signal pipeline before scaling.
• Prioritize calibration: A per-user baseline helps reduce inter-subject variance and improves model performance.
• Use spectral features: For repetitive, periodic movements, FFT-based features help separate controlled repetitions from erratic motion.
• Build non-blocking firmware early: Blocking code can silently destroy real-time performance, causing missed samples and jittery inference.
• Validate haptics with users: Haptic duration, intensity, and cooldowns must be tuned with actual gym users to avoid alarm fatigue.
• Preserve privacy by default: Local-only operation and clear privacy options increase user trust, which matters in fitness contexts.

What’s next for VibeCoach — roadmap items and research directions

Key priorities outlined by the prototype author:

• Dataset expansion to include more incorrect-form examples and a broader subject pool.
• BLE integration to stream telemetry and historical metrics to a companion app for progress tracking and coach review.
• Hardware miniaturization: migrate to an SMD PCB and ergonomic enclosure to make the device unobtrusive for everyday gym use.
• Personalized coaching: adaptive thresholds and tailored training programs that adjust sensitivity based on user experience level and fatigue patterns.
• Multi-exercise models: extend the TinyML pipeline to cover a range of compound and isolation movements, preserving on-device inference.

Longer-term research avenues

• Multi-node synchronization: combine multiple wearable nodes (e.g., chest + arm) to recover richer kinematic context without cameras.
• Transfer learning across exercises: use shared layers or embeddings to accelerate new-exercise training with less data.
• Adaptive feedback strategies: explore reinforcement learning for individualized correction timing and modality (haptic vs visual).

Final perspective on the prototype’s significance

VibeCoach demonstrates that precise, private, and immediate form correction is achievable on resource-constrained wearable hardware. The proof of concept shows how careful sensor fusion, per-user calibration, spectral feature extraction, and TinyML can deliver actionable real-time feedback without cameras or cloud services. Early results are promising: 90.1% validation accuracy on a focused task and a firmware architecture that preserves real-time sampling and inference. The next steps are centered on broadening data, refining user experience, and engineering the hardware for production. If those pieces come together, camera-free wearable coaching could make evidence-based technique correction widely available and affordable, helping more people train safer and more effectively.

FAQ

Q: How does VibeCoach know when form is wrong without a camera? A: The device uses a 9-axis IMU (accelerometer, gyroscope, magnetometer) to estimate orientation quaternions through the Madgwick filter. After a per-user baseline calibration (inverse quaternion transform), the system extracts spectral and temporal features from overlapping 1-second windows and runs a TinyML classifier locally to label motion as idle, correct_move, or wrong_move.

Q: Can VibeCoach replace a personal trainer? A: VibeCoach offers targeted, immediate correction for specific patterns like posture deviations during lateral raises. It does not replace a coach’s broader expertise—programming, progressive overload planning, injury diagnosis—but it complements coaching by providing consistent, private micro-feedback that a lifter can use between sessions with a trainer.

Q: Why no Bluetooth or smartphone integration yet? A: The prototype focuses on proving on-device inference, privacy, and real-time haptic feedback without network dependencies. BLE and companion apps are planned future enhancements to enable session logging, coach connectivity, and richer analytics.

Q: How accurate is the TinyML model? A: In the initial prototype trained on lateral raises, the model achieved 90.1% validation accuracy in Edge Impulse Studio. Idle detection was especially robust (96.7%). The main limitation is dataset size and diversity; expanding training data will improve generalization.

Q: Where should I place the device on my body? A: Placement depends on the exercise. For lateral raises, the prototype is intended for the upper arm to capture shoulder abduction and rotation. For other exercises like squats, the device could be placed on the chest to monitor torso inclination. The modular placement concept is a key design feature.

Q: Does the device record or transmit video or audio? A: No. VibeCoach is camera- and microphone-free by design. It collects only numeric orientation and motion data from the IMU, supporting privacy in public gym settings.

Q: How intrusive are the haptic alerts? A: Alerts are graded. A score-accumulator prevents single-frame noise from triggering feedback. Severity is mapped to pulse intensity and count with cooldowns to avoid annoyance. Calibration with users will further refine intensity levels.

Q: What battery life can I expect? A: The prototype uses a 250 mAh Li-Po battery, which supports prototyping sessions. Actual battery life depends on sampling frequency, vibration usage, and display activity. Firmware choices (running MCU at 50 MHz, non-blocking ADC sampling) were made to extend runtime, but production devices would require battery optimizations and likely larger capacity.

Q: Can the model handle different body sizes and slight variations in exercise style? A: The calibration routine reduces some variability by normalizing the starting pose, but model robustness increases with dataset diversity. Collecting more samples across body types and lifting styles is the primary path to generalize performance.

Q: Is the system suitable for heavy compound lifts? A: The current prototype is tuned for controlled, periodic movements like lateral raises. Heavy compound lifts introduce larger dynamic ranges and safety concerns; those exercises may require additional sensors, different placement (multiple nodes), and thorough validation before recommending automated haptic corrections.

Q: What are the main next steps before a commercial product? A: Expanding the training dataset, adding BLE for optional logging, designing a custom SMD PCB for miniaturization, rigorous battery and EMC testing, and refining ergonomics and UI flow to meet regulatory and consumer standards.

Q: How can researchers or creators contribute data or collaborate? A: Teams building similar systems benefit from shared movement datasets, labelled incorrect-form patterns, and validated protocols for safe haptic feedback. Collaboration can accelerate model generalization and create common benchmarks for IMU-based form correction.