Duplicate samples streaming to brainflow after toggling "start data stream"

[DashBarkHuss]

Problem

Streaming to brainflow works as expected when you start a session in OpenBCI GUI- until you toggle "Start Data Stream" off and back on. After this the stream shows duplicates. Once you end the session and start a new one the GUI will stream correctly again with only unique values.

UPDATE: Upon further review the data isn't only duplicated but it's not always in order either

Expected

Toggling "Start Data Stream" should not cause duplication of samples

Operating System and Version

macOS 15.5

GUI Version

v6.0.0 beta.1

Are you running the downloaded app or from Processing/VSCode?

Downloaded

Type of OpenBCI Board

Replicated with Synthetic and Cyton Daisy

To replicate

1. GUI settings Synthetic (algorithmic)
   16 chan
   Branflow: Network
   IP: 225.1.1
   Port: 6677

2. Start the session

3. Run the python script included at the bottom of this post

4. Press Start Data Stream for the first time for the session

5. Check logs for the python script you ran.
   You should see logs like this:

Batch   1:  18 samples, first: 1754072100.724668, last: 1754072100.793457, duplicates: 0
Batch   2: 126 samples, first: 1754072100.797082, last: 1754072101.297603, duplicates: 0
Batch   3: 126 samples, first: 1754072101.301602, last: 1754072101.801853, duplicates: 0
Batch   4: 126 samples, first: 1754072101.805660, last: 1754072102.305875, duplicates: 0
Batch   5: 126 samples, first: 1754072102.309604, last: 1754072102.809387, duplicates: 0
Batch   6:  33 samples, first: 1754072102.813971, last: 1754072102.941663, duplicates: 0

0 duplicates- All good!

6. But now toggle Stop/Start Data Stream off and on

Now the logs show duplicates

Batch   7: 195 samples, first: 1754072102.945605, last: 1754072133.563728, duplicates: 96
⚠️  FOUND 96 DUPLICATE TIMESTAMPS IN THIS BATCH!
   Example duplicates: [np.float64(1754072102.945605), np.float64(1754072133.17523), np.float64(1754072133.180317), np.float64(1754072133.17523), np.float64(1754072133.180317)]
Batch   8: 252 samples, first: 1754072133.559963, last: 1754072134.068253, duplicates: 126
⚠️  FOUND 126 DUPLICATE TIMESTAMPS IN THIS BATCH!
   Example duplicates: [np.float64(1754072133.559963), np.float64(1754072133.563728), np.float64(1754072133.567628), np.float64(1754072133.567628), np.float64(1754072133.572238)]
Batch   9: 255 samples, first: 1754072134.063950, last: 1754072134.575852, duplicates: 128
⚠️  FOUND 128 DUPLICATE TIMESTAMPS IN THIS BATCH!
   Example duplicates: [np.float64(1754072134.06395), np.float64(1754072134.068253), np.float64(1754072134.072089), np.float64(1754072134.072089), np.float64(1754072134.075733)]
Batch  10: 252 samples, first: 1754072134.575852, last: 1754072135.080416, duplicates: 126
⚠️  FOUND 126 DUPLICATE TIMESTAMPS IN THIS BATCH!

Temporary solution

Always start a new session or look into filtering out duplicates via code

Script

from brainflow.board_shim import BoardShim, BrainFlowInputParams, BoardIds
import time
import numpy as np
from collections import defaultdict

def main():
    print("OpenBCI Stream Duplicate Detector")
    print("=" * 50)
    
    # Board configuration
    BOARD_TYPE = BoardIds.CYTON_DAISY_BOARD
    
    # Setup board connection  
    BoardShim.enable_dev_board_logger()
    
    params = BrainFlowInputParams()
    params.ip_port = 6677
    params.ip_address = "225.1.1.1"
    params.master_board = BOARD_TYPE
    
    board = BoardShim(BoardIds.STREAMING_BOARD, params)
    
    try:
        # Connect and start streaming
        print("Connecting to OpenBCI stream...")
        board.prepare_session()
        board.start_stream()
        
        # Get board info
        timestamp_channel = BoardShim.get_timestamp_channel(BOARD_TYPE)
        print(f"Connected! Timestamp channel: {timestamp_channel}")
        print("\nMonitoring for duplicate timestamps...")
        print("(Try stopping/starting data stream in OpenBCI GUI to reproduce issue)")
        print("-" * 60)
        
        # Wait for buffer to fill
        time.sleep(2)
        
        # Track all timestamps we've seen
        seen_timestamps = set()
        total_samples = 0
        duplicate_count = 0
        batch_count = 0
        
        # Stream monitoring loop
        while True:
            # Get all new data since last call
            data = board.get_board_data()
            
            if data.size > 0:
                batch_count += 1
                timestamps = data[timestamp_channel]
                num_samples = len(timestamps)
                total_samples += num_samples
                
                # Check for duplicates in this batch
                batch_duplicates = 0
                for ts in timestamps:
                    if ts in seen_timestamps:
                        batch_duplicates += 1
                        duplicate_count += 1
                    else:
                        seen_timestamps.add(ts)
                
                # Report batch info
                first_ts = timestamps[0] if len(timestamps) > 0 else 0
                last_ts = timestamps[-1] if len(timestamps) > 0 else 0
                
                print(f"Batch {batch_count:3d}: {num_samples:3d} samples, "
                      f"first: {first_ts:.6f}, last: {last_ts:.6f}, "
                      f"duplicates: {batch_duplicates}")
                
                # Alert on duplicates
                if batch_duplicates > 0:
                    print(f"⚠️  FOUND {batch_duplicates} DUPLICATE TIMESTAMPS IN THIS BATCH!")
                    
                    # Show some example duplicates
                    duplicate_examples = []
                    for ts in timestamps:
                        if ts in seen_timestamps and len(duplicate_examples) < 5:
                            duplicate_examples.append(ts)
                    
                    if duplicate_examples:
                        print(f"   Example duplicates: {duplicate_examples}")
                
                # Summary every 20 batches
                if batch_count % 20 == 0:
                    print(f"\n📊 Summary: {total_samples} total samples, {duplicate_count} duplicates ({duplicate_count/total_samples*100:.1f}%)")
                    if duplicate_count > 0:
                        print("🔍 Try stopping/starting the data stream in OpenBCI GUI to see if duplicates increase")
                    print("-" * 60)
                    
            else:
                print("No data received...")
            
            time.sleep(0.5)  # Check every 0.5 seconds
            
    except KeyboardInterrupt:
        print(f"\n\n📈 Final Results:")
        print(f"Total samples processed: {total_samples}")
        print(f"Duplicate timestamps found: {duplicate_count}")
        if total_samples > 0:
            print(f"Duplicate rate: {duplicate_count/total_samples*100:.2f}%")
        
        if duplicate_count > 0:
            print("\n🚨 Duplicates detected! This indicates an issue with:")
            print("   - BrainFlow buffer management, or")
            print("   - OpenBCI GUI streaming behavior, or") 
            print("   - Network streaming protocol")
        else:
            print("\n✅ No duplicates detected - streaming appears clean")
            
    except Exception as e:
        print(f"Error: {e}")
        
    finally:
        # Cleanup
        if board.is_prepared():
            board.stop_stream()
            board.release_session()
        print("Stream stopped.")

if __name__ == "__main__":
    main()