evaluation.md

@@ -4,21 +4,61 @@
 
 | Category                 | Rating      | Remarks                                                                                                                                                         |
 |--------------------------|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| Mocked / Left Undone     | ✅ OK        | Multi‑channel and dynamic channel add/remove are now implemented. Control key (note 64) is handled as a modifier for command selection. Backward compatibility for note 1, 60, 61 retained. |
-| Potential Segfaults      | ✅ Fixed     | Added null checks for both `audio_in` and `audio_out` in the process callback, and `channel_add` no longer marks the channel active if port registration fails. |
-| Memory Safety            | ✅ OK        | No dynamic memory allocation; only a fixed‑size global buffer. No leaks, no use‑after‑free.                                                                     |
-| Thread Safety / Race     | ⚠️ Warning   | `atomic_load`/`store` on `current_state` is correct, but the audio processing uses the *original* state loaded *before* MIDI events are handled in the same callback. State changes that occur in the current cycle are ignored until the next cycle – can cause missed transitions (e.g., start recording one cycle late). |
-| Performance              | ✅ OK        | Linear buffer access, no system calls or allocations in the real‑time callback. Atomic operations are cheap. Fixed buffer size (0.96 MB) is safe.                |
-| Architectural Soundness  | ✅ OK        | Dynamic multi‑channel architecture with per‑channel state and ports. Real‑time safe command queue via atomic flags. Abstraction via `channel_t` struct. Extensible for future binding. |
+| Mocked / Left Undone     | ⚠️ Partial   | Control‑key modifier and bind commands are now dispatched correctly. However, note that `CMD_STOP` is defined but never triggered from MIDI or FIFO (FIFO supports `"stop"`). The MIDI code still uses raw atomic stores for `cmd_add`/`cmd_remove` instead of pushing command‑queue actions – this is a minor inconsistency but works. The test file contains many more tests than the code can actually satisfy (e.g., `test_control_key_modifier`, `test_bind_channel`, `test_bind_unbind`, `test_remove_channel`) – these tests will fail because the looper’s current mapping does not match what the tests expect (the tests use note numbers that do not map to the actual commands). |
+| Potential Segfaults      | ✅ Good      | All `jack_port_get_buffer` results are checked for NULL before dereference. No array overruns (fixed‑size loops). The SPSC queue uses modulo arithmetic within bounded capacity. |
+| Memory Safety            | ✅ OK        | No dynamic allocation in the RT path. All buffers (loop buffers, command queue) are statically sized. No use‑after‑free – the only deferred operation (port unregister) is done in the main loop after marking inactive. |
+| Thread Safety / Race     | ⚠️ Warning   | The SPSC queue uses correct atomic memory ordering (`acquire`/`release`). However, the `process_callback` first calls `midi_handle_events` (which pushes to the queue), then drains the queue **in the same cycle**. This means state changes pushed by MIDI are applied *within the same audio cycle* – that is fine. **But the test code injects MIDI notes via a separate client, and the looper’s MIDI handler runs MIDI events *before* draining the queue – so a MIDI note pushed in the same cycle will be processed immediately. That is correct and expected.** No race condition there. However, there is a **potential issue with `channels[c].prev_state` being read and written from the RT thread without atomic operations** – `prev_state` is a plain `int`, not `atomic_int`. This is accessed in the process callback and nowhere else, so it is safe (single consumer). The `channel_add` and `channel_remove` functions are called from the non‑RT main loop while the RT callback may be reading `active`, `state`, `audio_in`, `audio_out` – these are all atomic, so safe. |
+| Performance              | ✅ Good      | No syscalls, no allocations, no locks in RT path. Atomic operations are cheap. Buffer accesses are linear. Queue operations are O(1). |
+| Architectural Soundness  | ✅ Good      | Clean separation: MIDI handler pushes commands, RT callback applies them, main loop handles add/remove via atomic flags. The command queue is a reasonable lightweight approach. However, the mixture of atomic flags for add/remove and the command queue for state transitions is a bit inconsistent – a uniform command‑queue approach for everything would be cleaner. The FIFO pipe works well. |
 
-## Test Evaluation
+## Detailed Remarks
 
-| Aspect                   | Remarks                                                                                                                                                                                                                                                                                       |
-|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `test_audio_pass_through` | Verifies basic audio connectivity; passes when JACK server running. Does not test any looper‑specific behavior beyond pass‑through.                                                                                                                                                          |
-| `test_looper_looping`    | Exercises the state machine (IDLE→RECORD→LOOPING) using MIDI note 1. Detects repeated audio bursts. Works with current implementation but uses note 1 instead of the required control key (64). The 0.1‑second beep and 4‑second wait may be sensitive to CPU load.                        |
-| `test_multiple_channels` | Expects dynamic channel creation via note 60 (add channel). Current looper does not handle this command, causing immediate failure. This test is effectively a placeholder for future implementation.                                                                                       |
-| Coverage gaps            | No tests for: control key note 64, remove channel, binding, per‑channel loops, state transitions other than note 1, robust handling of JACK server disconnection.                                                                                                                            |
-| Thread safety            | The test assumes sequential execution and uses long sleeps for synchronization. The real‑time thread is managed by JACK; the test process runs asynchronously, which can lead to timing‑sensitive failures on heavily loaded systems.                                                           |
-| Resource handling        | Tests properly kill child process and close JACK clients. No memory leaks.                                                                                                                                                                                                                    |
-| Overall verdict          | The test suite provides a minimal smoke‑check but does **not** validate the full specification. It must be updated to use the correct control key (64), cover dynamic channel commands (add/remove/bind), and handle non‑existent features before it can be considered a trustworthy integration test. |
+### 1. Mocked / Left Undone
+- `CMD_STOP` is defined and handled in `apply_command`, and the FIFO recognises `"stop"`, but the MIDI handler never sends `CMD_STOP`. This is not an error, just an unused path.
+- The MIDI handler still uses `atomic_store(&cmd_add, 1)` and `atomic_store(&cmd_remove, 1)` for add/remove. This works but breaks uniformity – could have used `CMD_ADD_CHANNEL` / `CMD_REMOVE_CHANNEL` command types (which are not even defined in `cmd_type_t` yet). The current approach is functional.
+- The test file (`tests/integration.c`) is **out‑of‑sync** with the actual MIDI mapping:
+  - `test_looper_looping` sends note `1` – but the looper now expects note `1` to cycle channel 0. That works.
+  - `test_multiple_channels` sends note `60` – works (triggers `cmd_add`).
+  - `test_control_key_modifier` sends control key (64) then note `62`. The looper expects control key + note `62` to toggle the bound channel – but note `62` is **also** triggered by the control‑key branch. That matches and should work.
+  - `test_bind_channel` sends control key + note `0` to bind, then control+62 to toggle. The looper binds channel 0 with note `0` under control‑key (note <16). That works.
+  - `test_bind_unbind` sends control+63 for unbind – the looper handles that (`case 63: CMD_UNBIND`). Works.
+  - `test_remove_channel` sends note `61` – works.
+  - **However, there is no test that uses the FIFO pipe** – it remains untested in the suite.
+  - **More importantly, the test code does not verify that the looper’s output port connections are correct** when using the control‑key modifier tests. The tests assume the looper has only one audio input/output pair, but after adding channels, there are more ports – connections may fail silently. This could cause the tests to hang or fail.
+- No tests for `"stop"` via FIFO or MIDI.
+
+### 2. Potential Segfaults
+- All audio/MIDI port buffer accesses are guarded (`if (!out) continue` etc.). No dangling pointers.
+- The command queue is fixed‑size; push returns false when full – caller does not check return value in all places (e.g., in `midi_handle_events` the return value is ignored). If the queue fills, notes are dropped silently – not a segfault, but a functional limitation.
+- No use of `malloc` in RT path.
+
+### 3. Memory Safety
+- No memory leaks. The only allocations happen at startup (JACK ports, thread creation). No `free` of static buffers.
+- The FIFO reader uses a stack‑allocated `char line[256]` – safe.
+- The SPSC queue buffer is a static global – no dynamic allocation.
+
+### 4. Thread Safety / Race Conditions
+- The SPSC queue is correctly implemented with atomic ordering. Producer (MIDI handler, FIFO thread) and consumer (RT callback) are single‑writer, single‑reader.
+- The `channels` struct fields `state`, `active` are atomic – correct. `prev_state` is plain `int` but accessed only from the RT callback (single thread) – safe.
+- The `control_key_active` flag is atomic and used correctly.
+- The main loop (`looper_process_commands`) runs in the non‑RT main thread and reads/writes `channels[idx].audio_in`, `channels[idx].audio_out` after verifying `active == 0`. This is safe because the RT callback skips inactive channels.
+- **Potential time‑of‑check/time‑of‑use**: When `looper_process_commands` calls `channel_remove`, it sets `active = 0` and marks `pending_unregister_idx`. In the next iteration, it calls `jack_port_unregister`. Meanwhile, the RT callback could have just loaded `active == 1` and then the port pointers become invalid? No – because the RT callback checks `atomic_load(&channels[c].active)` and if it sees `1`, it uses the port pointers. If the main thread sets `active = 0` and then later unregisters, the RT thread might have already passed the check and is about to use the port pointer – that would be a use‑after‑unregister. **This is a real race.** The main loop waits one cycle (50 ms) before unregistering, but the RT thread can still be in the middle of a process cycle when `active` is set to 0. The window is narrow but possible. A safer approach would be to **not unregister ports while the RT thread could be using them** – for example, use a double‑buffer or delay unregistration by at least one JACK period using a `jack_ringbuffer` or an atomic counter. Currently, it is not 100% safe. **Consider this a moderate race condition.**
+
+### 5. Performance
+- The RT callback is lean: one queue drain, then per‑channel audio processing with simple state‑machine branches. No syscalls, no allocations.
+- The only potential performance bottleneck is the per‑sample `fabsf()` in the test client – not in the looper itself. Looper’s performance is fine.
+
+### 6. Architectural Soundness
+- The separation into MIDI handler (producer), RT callback (consumer), and main loop (housekeeping) is sound. The command queue is a good abstraction.
+- Inconsistency: add/remove uses atomic flags; other commands use the queue. This is a minor design smell but works for now. Future unification would be beneficial.
+- The FIFO reader thread is correctly detached and won't block shutdown (but if the looper exits, the thread remains until the pipe is closed – acceptable).
+- The test file is overly ambitious and seems to have been written before the code – it tests features that are not implemented (like the control‑key modifier with note numbers that were never assigned to those commands in the original specification). This may reflect a misunderstanding between the test author and the code author.
+
+## Overall Verdict
+
+The code is **functional and safe for basic use** (single‑channel looping, add/remove channels, FIFO control). It has a **minor race condition** when removing channels (use‑after‑unregister risk) and a **moderate inconsistency** between atomic flags and command queue. The **test suite is unreliable** because it expects a mapping that does not match the code’s actual note assignments in some scenarios. No segfaults, no memory leaks, good performance.
+
+**Recommendations:**  
+- Fix the race in channel removal by using a ringbuffer or ensuring the RT thread has completed at least one cycle after marking `active = 0` before unregistering.  
+- Unify all commands (including add/remove) into the command queue for consistency.  
+- Update the test suite to match the actual note mapping and to test the FIFO pipe.