ServerNode

Main servernode software hub

Features

ServerNode is a highly-flexible, multi-client, high-channel count recording software and session management system, allowing multiple users to stream and record independent subsets of available data from an eCube unit.

An example setup with a central ServerNode process handling offline data saving, while multiple clients (networked or local) process data streams online.

Starting ServerNode

ServerNode.exe can be launched directly and should auto-detect a powered-on eCube if network setup has fully completed. To learn more about network setup in eCubeNetworkSetup, see the more detailed section.

If autodetection did not successfully complete, you may see this prompt, or a console that automatically self-exit. If you encounter this, please visit network setup (above), or in specialized network setups, you may way to pre-configure connection options.

PS C:\Users\eCubePC\Desktop\ServerNode-v0.2.3> .\servernode.exe
No hostname specified or detected.

  C:\Users\eCubePC\Desktop\ServerNode-v0.2.3\servernode.exe [--help]
    [--path <path>] [--seconds <size>] [--verbose] [--log <file>] [hostname]

    White Matter ServerNode v0.2.3
    Command line version of ServerNode that allow multiple clients to connect to
    a single ECube device or server.

  OPTIONS:

      -h, --help                        Display this help menu
      -d[path], --path=[path]           Directory to which to record data.
      -s[size], --seconds=[size]        Size of recording chunks in seconds.
      -v, --verbose                     Enable verbose logging
      -l[file], --log=[file]            Write log to file.
      hostname                          Name or IP of device or server to which
                                        to connect.
      "--" can be used to terminate flag options and force all following
      arguments to be treated as positional options

Pre-configuring options

ServerNode can be started with pre-configured data recording path using the -d or --path= option, and pre-configured recording chunk time using the -s or --seconds= option.

In specialized network setups, such as multiple eCubes or instances of ServerNode running on the same internal network, you may need to define the hostname explicitly. To find the hostname for the eCube under windows, look underneath the eCube unit, which should contain a label of the format:

ecube-xxxx.mshome.net

Where xxxx refers to a specific alphanumeric serial for your specific eCube unit. You may then launch ServerNode with these pre-set options. For example, configuring a Windows shortcut with the command:

C:\path\to\servernode.exe -d D:\data\20201001\ -s 300 ecube-C83C.mshome.net

Will start a ServerNode instance that saves upcoming recorded data to D:\data\20201001\ with 5-minute seamless auto-segmenting data chunks, and connects to the explicit ecube unit C83C.

To pre-configure these settings every time on startup, use a Powershell script or create a shortcut with these options.

Headstage management

Once started, ServerNode is in headstage management mode. ServerNode must have all headstages for immediate recording session use added prior to starting headstage streaming.

To add or remove headstage channels, use:

Listing connected headstages

The command list may be used to list available and connected headstages. Each headstage will report its connected position, and the number of available channels.

Note that hs-640 headstages will always present 640 available channels.

> list
Available modules and channels:
    - Headstage 2 (64 channels)
    - Headstage 3 (64 channels)
    - Headstage 8 (640 channels)
    - Headstage 9 (640 channels)
    - Headstage 10 (640 channels)
    - Analog Panel (32 channels)
- Digital Panel (64 channels)

Adding / removing connected headstages

Adding headstages takes the syntax of:

add [headstage number] [number of channels]

For example, a stackable headstage plugged into the slot H8, with 4 boards consituting 256 channels:

add 8 256

Adding multiple headstages might look like this:

> add 2

> add 8 512

> add 9 512

Selected headstage channels:
- Headstage 2: 64
- Headstage 8: 512
- Headstage 9: 512

Starting headstages

Headstages are streamed using the command start. Any connected clients, including Open Ephys and ServerNode-control, will have access to headstage data after starting.

Please note that headstage management is unavailable after the start command is issued. To start a synchronized large recording with a different set of headstages, terminate and restart the ServerNode program.

Multi-headstage coordination

An eCube unit coordinates the synchronized streaming of all its connected headstages, and allows for the unique feature of unplugging / replugging of headstages even during streaming and data recording.

This allows for complex experiments, with multiple experimenters managing independent subsets of headstage and Analog / Digital panel inputs, as well as immediate recovery during behaving animal experiments where free behavior may temporarily unplug the magnetic self-aligning headstage.

An unplugged headstage is represented by 0 V signals on all its channels, until the headstage is replugged. When the headstage is replugged, all headstages re-synchronize with each other. This causes a brief 100-250ms data gap in all active headstages. The ServerNode recording software handles this behavior, and makes sure that all actively recording files are correctly padded with data samples for synchronization between all headstages and digital / analog panel inputs.

Faulty-tether rejection

To prevent a faulty tether (e.g., from experimental animal damage) from repeatly resetting all headstages and introducing data gaps in other experimental animal recordings, the eCube implements something known as faulty-tether rejection.

After a number of detected apparent unplug/replugs in a short period of time (5 resets in 120 seconds), the headstage tether will be blocked from reconnection.

This blocking state can be managed through ServerNode’s tether command. Issuing:

tether --reset

will immediately reset all blocked headstages, and allow headstage replug to continue.

Disabling the rejection

For working on-the-bench and testing electrodes where headstage replugs or resets may occur frequently, it is sometimes desirable to disable this feature entirely. To do this, issue:

tether --disable

will entirely disable the faulty-tether rejection feature. This can be re-enabled in the future for critical multi-subject recordings using tether --enable.

Power-cycling (turning off and on) the eCube will reset this feature to the default enabled status.

Enabling low latency modes (beta)

This feature requires firmware 20210726 or higher, and ServerNode / OpenEphys eCube plugin v0.4.0-beta or higher.

By default, each headstage data packet obtained from the eCube will contain 728 samples, corresponding to 29 ms of continuous data from all headstages. This default mode is our recommended data packet streaming size for most experiments, which supports the highest headstage channel count.

If you have a real-time neural experiment requiring lower latency and a higher update rate, you may put the eCube in a low latency mode using the stream option in servernode. This means eCube will send out headstage data at a significantly higher packet rate, with a tradeoff of a smaller supported neural channel count.

The following table describes the specifications of low latency modes:

Mode Command Samples / packet Maximum supported channels Est. full-trip latency to arbitrary sample
Factory (default mode after eCube boot) 728 (29 ms of data) ~2000 at full 25kSps samplerate 31±16 ms @ 1280 ch
LL-1 stream --lowlatency-1 384 (15 ms) 1280 channels 18±9 ms @ 1280 ch
LL-2 stream --lowlatency-1 160 (6 ms) 512 channels 9±5 ms @ 512 ch

The stream command in servernode can be used to change a streaming mode:

> stream
Current session is 728 samples / packet.

Invalid streaming data package size command.
  > stream {OPTIONS}

    Control data streaming configurations.

  Supported commands:

      Streaming data package size
      control:
        --factory                         Factory: 728 samples (29 ms) / packet
        --lowlatency-1                    Low latency: 384 samples (15 ms) /
                                          packet
        --lowlatency-2                    Low latency: 160 samples (6 ms) /
                                          packet

To set a low latency mode, before starting data streaming from an eCube, issue the command in servernode:

> stream --lowlatency-1
eCube set to low-latency 384 samples (15 ms) / packet.

Note: Once you use servernode, servernode-control, a plugin, or pyeCubeStream.py to start streaming eCube data, the latency mode becomes fixed and should not be changed without first rebooting the eCube.

To use a low latency streaming data mode, please do so on a fresh boot of the eCube, before data is streamed.

Downstream clients, such as servernode-control and the eCube Open Ephys plugin, will automatically detect the streaming mode from upstream servernode.


Last modified July 29, 2021