Skip to content

Commit e41bf27

Browse files
committed
first pass of installation setup
1 parent 80e497c commit e41bf27

29 files changed

+1376
-694
lines changed

.gitignore

+8-4
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
1-
code/data/
2-
code/__pycache__/
3-
code/signalstore
1+
*.npy
2+
*/*.npy
3+
*/*/*.npy
4+
eegdash/data/
5+
*/__pycache__/
46
.venv
5-
.env
7+
.secret
8+
tests/.testeegdash/
9+

LICENSE

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
GNU General Public License
2+
3+
Copyright (C) 2024-2025
4+
5+
Young Truong, UCSD, [email protected]
6+
Arnaud Delorme, UCSD, [email protected]
7+
8+
This program is free software; you can redistribute it and/or modify
9+
it under the terms of the GNU General Public License as published by
10+
the Free Software Foundation; either version 2 of the License, or
11+
(at your option) any later version.
12+
13+
This program is distributed in the hope that it will be useful,
14+
but WITHOUT ANY WARRANTY; without even the implied warranty of
15+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16+
GNU General Public License for more details.
17+
18+
You should have received a copy of the GNU General Public License
19+
along with this program; if not, write to the Free Software
20+
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1.07 USA

code/__init__.py

Whitespace-only changes.
Binary file not shown.

code/test_signalstore.ipynb

-689
This file was deleted.

dist/eegdash-0.0.1-py3-none-any.whl

31.8 KB
Binary file not shown.

dist/eegdash-0.0.1.tar.gz

29.8 KB
Binary file not shown.

eegdash.egg-info/PKG-INFO

+72
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
Metadata-Version: 2.1
2+
Name: eegdash
3+
Version: 0.0.1
4+
Summary: EEG data for machine learning
5+
Author-email: Young Truong <[email protected]>, Arnaud Delorme <[email protected]>
6+
License: GNU General Public License
7+
8+
Copyright (C) 2024-2025
9+
10+
Young Truong, UCSD, [email protected]
11+
Arnaud Delorme, UCSD, [email protected]
12+
13+
This program is free software; you can redistribute it and/or modify
14+
it under the terms of the GNU General Public License as published by
15+
the Free Software Foundation; either version 2 of the License, or
16+
(at your option) any later version.
17+
18+
This program is distributed in the hope that it will be useful,
19+
but WITHOUT ANY WARRANTY; without even the implied warranty of
20+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
21+
GNU General Public License for more details.
22+
23+
You should have received a copy of the GNU General Public License
24+
along with this program; if not, write to the Free Software
25+
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1.07 USA
26+
27+
Project-URL: Homepage, https://github.com/sccn/EEG-Dash-Data
28+
Project-URL: Issues, https://github.com/sccn/EEG-Dash-Data/issues
29+
Classifier: Programming Language :: Python :: 3
30+
Classifier: License :: GNU General Public License
31+
Classifier: Operating System :: OS Independent
32+
Requires-Python: >=3.8
33+
Description-Content-Type: text/markdown
34+
License-File: LICENSE
35+
36+
# EEG-Dash
37+
To leverage recent and ongoing advancements in large-scale computational methods and to ensure the preservation of scientific data generated from publicly funded research, the EEG-DaSh data archive will create a data-sharing resource for MEEG (EEG, MEG) data contributed by collaborators for machine learning (ML) and deep learning (DL) applications.
38+
39+
## Data source
40+
The data in EEG-DaSh originates from a collaboration involving 25 laboratories, encompassing 27,053 participants. This extensive collection includes MEEG data, which is a combination of EEG and MEG signals. The data is sourced from various studies conducted by these labs, involving both healthy subjects and clinical populations with conditions such as ADHD, depression, schizophrenia, dementia, autism, and psychosis. Additionally, data spans different mental states like sleep, meditation, and cognitive tasks. In addition, EEG-DaSh will also incorporate data converted from NEMAR, which includes a subset of the 330 MEEG BIDS-formatted datasets available on OpenNeuro, further expanding the archive with well-curated, standardized neuroelectromagnetic data.
41+
42+
## Data formatting
43+
The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
44+
45+
![Screenshot 2024-10-03 at 09 07 28](https://github.com/user-attachments/assets/b30a79bb-0d94-410a-843c-44c3fcea01fc)
46+
47+
## Data access
48+
The data in EEG-DaSh is formatted to facilitate machine learning (ML) and deep learning (DL) applications by using a simplified structure commonly adopted by these communities. This will involve converting raw MEEG data into a matrix format, where samples (e.g., individual EEG or MEG recordings) are represented by rows, and values (such as time or channel data) are represented by columns. The data is also divided into training and testing sets, with 80% of the data allocated for training and 20% for testing, ensuring a balanced representation of relevant labels across sets. Hierarchical Event Descriptor (HED) tags will be used to annotate labels, which will be stored in a text table, and detailed metadata, including dataset origins and methods. This formatting process will ensure that data is ready for ML/DL models, allowing for efficient training and testing of algorithms while preserving data integrity and reusability.
49+
50+
The data in EEG-DaSh is accessed through Python and MATLAB libraries specifically designed for this platform. These libraries will use objects compatible with deep learning data storage formats in each language, such as <i>Torchvision.dataset</i> in Python and <i>DataStore</i> in MATLAB. Users can dynamically fetch data from the EEG-DaSh server which is then cached locally.
51+
52+
### AWS S3
53+
54+
Coming soon...
55+
56+
### EEG-Dash API
57+
58+
Coming soon...
59+
60+
## Education
61+
62+
We organize workshops and educational events to foster cross-cultural education and student training, offering both online and in-person opportunities in collaboration with US and Israeli partners. There is no event planned for 2024. Events for 2025 will be advertised on the EEGLABNEWS mailing list so make sure to [subscribe](https://sccn.ucsd.edu/mailman/listinfo/eeglabnews).
63+
64+
## About EEG-DaSh
65+
66+
EEG-DaSh is a collaborative initiative between the United States and Israel, supported by the National Science Foundation (NSF). The partnership brings together experts from the Swartz Center for Computational Neuroscience (SCCN) at the University of California San Diego (UCSD) and Ben-Gurion University (BGU) in Israel.
67+
68+
![Screenshot 2024-10-03 at 09 14 06](https://github.com/user-attachments/assets/327639d3-c3b4-46b1-9335-37803209b0d3)
69+
70+
71+
72+

eegdash.egg-info/SOURCES.txt

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
LICENSE
2+
README.md
3+
pyproject.toml
4+
setup.py
5+
eegdash/__init__.py
6+
eegdash/aws_ingest.py
7+
eegdash/data_utils.py
8+
eegdash/signalstore_data_utils.py
9+
eegdash.egg-info/PKG-INFO
10+
eegdash.egg-info/SOURCES.txt
11+
eegdash.egg-info/dependency_links.txt
12+
eegdash.egg-info/top_level.txt
13+
eegdash/signalstore/__init__.py
14+
eegdash/signalstore/operations/__init__.py
15+
eegdash/signalstore/operations/handler_executor.py
16+
eegdash/signalstore/operations/handler_factory.py
17+
eegdash/signalstore/store/__init__.py
18+
eegdash/signalstore/store/data_access_objects.py
19+
eegdash/signalstore/store/datafile_adapters.py
20+
eegdash/signalstore/store/repositories.py
21+
eegdash/signalstore/store/store_errors.py
22+
eegdash/signalstore/store/unit_of_work.py
23+
eegdash/signalstore/store/unit_of_work_provider.py
24+
tests/__init__.py

eegdash.egg-info/dependency_links.txt

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+

eegdash.egg-info/top_level.txt

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
eegdash
2+
tests
File renamed without changes.

eegdash/__init__.py

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
from os import path
2+
import sys
3+
sys.path.append(path.abspath('../eegdash'))
4+
from eegdash import signalstore
5+
from .signalstore_data_utils import *
6+
from .data_utils import *

code/aws_ingest.py renamed to eegdash/aws_ingest.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import sys
22
sys.path.append('..')
33
import argparse
4-
from signalstore_data_utils import SignalstoreBIDS
4+
from eegdash.signalstore_data_utils import SignalstoreBIDS
55

66
def add_bids_dataset(args):
77
signalstore_aws = SignalstoreBIDS(
File renamed without changes.
File renamed without changes.

code/signalstore_data_utils.py renamed to eegdash/signalstore_data_utils.py

+14
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,20 @@ def find(self, query:dict, validate=False, get_data=False):
252252
else:
253253
return []
254254

255+
def get(self, query:dict, validate=False):
256+
'''
257+
query: {
258+
'dataset': 'dsxxxx',
259+
260+
}'''
261+
with self.uow as uow:
262+
sessions = uow.data.find(query, validate=validate, get_data=True)
263+
if sessions:
264+
print(f'Found {len(sessions)} records')
265+
return sessions
266+
else:
267+
return []
268+
255269
if __name__ == "__main__":
256270
# sstore_hbn = SignalstoreHBN()
257271
# sstore_hbn.add_data()

pyproject.toml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
[build-system]
2+
requires = ["setuptools>=61.0"]
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "eegdash"
7+
version = "0.0.1"
8+
authors = [
9+
{ name="Young Truong", email="[email protected]" },
10+
{ name="Arnaud Delorme", email="[email protected]" },
11+
]
12+
description = "EEG data for machine learning"
13+
readme = "README.md"
14+
license = { file = "LICENSE" }
15+
requires-python = ">=3.8"
16+
classifiers = [
17+
"Programming Language :: Python :: 3",
18+
"License :: GNU General Public License",
19+
"Operating System :: OS Independent",
20+
]
21+
22+
[project.urls]
23+
Homepage = "https://github.com/sccn/EEG-Dash-Data"
24+
Issues = "https://github.com/sccn/EEG-Dash-Data/issues"

requirements.txt

+18
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,15 @@ aiohappyeyeballs==2.4.3
33
aiohttp==3.10.10
44
aioitertools==0.12.0
55
aiosignal==1.3.1
6+
appnope==0.1.4
7+
asciitree==0.3.3
68
asttokens==2.4.1
79
async-timeout==4.0.3
810
attrs==24.2.0
911
botocore==1.35.36
12+
build==1.2.2.post1
1013
certifi==2024.8.30
14+
cftime==1.6.4.post1
1115
charset-normalizer==3.4.0
1216
comm==0.2.2
1317
contourpy==1.3.0
@@ -17,10 +21,14 @@ decorator==5.1.1
1721
dnspython==2.7.0
1822
exceptiongroup==1.2.2
1923
executing==2.1.0
24+
fasteners==0.19
2025
fonttools==4.54.1
2126
frozenlist==1.5.0
2227
fsspec==2024.10.0
28+
h5netcdf==1.4.0
29+
h5py==3.12.1
2330
idna==3.10
31+
iniconfig==2.0.0
2432
ipykernel==6.29.5
2533
ipython==8.28.0
2634
jedi==0.19.1
@@ -39,32 +47,40 @@ matplotlib-inline==0.1.7
3947
mne==1.8.0
4048
multidict==6.1.0
4149
nest-asyncio==1.6.0
50+
netCDF4==1.7.2
51+
numcodecs==0.13.1
4252
numpy==2.1.2
4353
packaging==24.1
4454
pandas==2.2.3
4555
parso==0.8.4
4656
pexpect==4.9.0
4757
pillow==11.0.0
4858
platformdirs==4.3.6
59+
pluggy==1.5.0
4960
pooch==1.8.2
5061
prompt_toolkit==3.0.48
5162
propcache==0.2.0
5263
psutil==6.1.0
5364
ptyprocess==0.7.0
5465
pure_eval==0.2.3
5566
Pygments==2.18.0
67+
pymatreader==1.0.0
5668
pymongo==4.10.1
5769
pyparsing==3.2.0
70+
pyproject_hooks==1.2.0
71+
pytest==8.3.3
5872
python-dateutil==2.9.0.post0
5973
pytz==2024.2
6074
pyzmq==26.2.0
6175
referencing==0.35.1
6276
requests==2.32.3
6377
rpds-py==0.20.0
6478
s3fs==2024.10.0
79+
scikit-learn==1.5.2
6580
scipy==1.14.1
6681
six==1.16.0
6782
stack-data==0.6.3
83+
threadpoolctl==3.5.0
6884
tornado==6.4.1
6985
tqdm==4.66.5
7086
traitlets==5.14.3
@@ -74,4 +90,6 @@ urllib3==2.2.3
7490
wcwidth==0.2.13
7591
wrapt==1.16.0
7692
xarray==2024.9.0
93+
xmltodict==0.14.2
7794
yarl==1.16.0
95+
zarr==2.18.3

tests/__init__.py

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
from os import path
2+
import sys
3+
sys.path.append(path.abspath('../eegdash'))
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)