-
Notifications
You must be signed in to change notification settings - Fork 13
Added environment setup instructions #181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
f03cf08
266a4c5
a14ae43
dd98ee9
86470dd
aab3bcf
18884ad
06ac6dd
f72f27c
cce88b8
9405c8c
aed3ffa
ce04198
2cfc014
4c53e27
13ca473
3c1f16a
6b74925
9387954
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| .. _Set_up_Python: | ||
|
|
||
| Set up Python | ||
| ============= | ||
|
|
||
| Please follow the instructions below to set up an appropriate environment to use PyHyperScattering. All of these setup steps should be run in a terminal, not a Jupyter notebook. | ||
|
|
||
| These instructions have been tested for the use of PyHyperScattering in a local JupyterLab notebook using the Anaconda distribution (https://www.anaconda.com/download) on a Windows computer. The instructions below *might* work for other platforms (e.g., NSLS II Jupyterhub, Google Colab), but there are no guarantees; recently, the NSLS II Jupyterhub has been especially incompatible PyHyperScattering. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A reference to a specific facility's platform is entirely inappropriate to have here. Also, simply setting this incompatibility in stone is unproductive; it would be prudent for NSLS2 staff to actually engage and make this work on their systems, especially with recent fixes to dependency management, things might work a lot better.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, it is worth mentioning that Colab and Jupyterhub are container-based platforms but you do not really have shell access (or you only have them inside the container) so I wouldn't offhand expect the command line parts to work.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. also, also we shouldn't be endorsing a commercial product like Anaconda, miniforge is probably what I would recommend, or refer out to somebody else who can endorse a product |
||
|
|
||
| Download Git | ||
| ------------ | ||
|
|
||
| To aid this workflow, download Git (https://git-scm.com/download/win). Then in the command prompt (not Anaconda Prompt), run ``winget install --id Git.Git -e --source winget``. After this, if you are able to run ``git --version`` and have a version number outputted, the installation was successful. If Anaconda Prompt was open, it may need to be restarted. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As I said in the comment chain, if the user is going to use git - they don't need it unless they plan to develop - they should really use GitHub desktop which is a lot easier. |
||
|
|
||
| Create and activate an environment | ||
| ---------------------------------- | ||
|
|
||
| - Open the Anaconda Prompt. Do not use the terminal feature after opening JupyterLab. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Idk what the second sentence means? Do not use the terminal within jupyterlab? I don't offhand know why. |
||
|
|
||
| - Identify the current conda environment, which appears in parentheses at the beginning of the command prompt. If a conda environment that is not base and is not the desired environment is active, deactivate the conda environment. Do not deactivate the base environment. | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| conda deactivate | ||
|
|
||
| - Create a new environment if a suitable one does not exist. Replace ``YOUR_ENVIRONMENT_NAME`` with a an environment name of choice that does not contain any spaces. If needed, the ``...`` can be replaced with other conda packages to be installed in this environment. The current notebook was run with the ``...`` omitted. After loading some packages, you will be asked if you want to proceed. Enter y (yes). | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| conda create -n YOUR_ENVIRONMENT_NAME ipykernel ... | ||
|
|
||
| If there already is an appropriate environment, skip this step. The purpose of the conda environment is to contain the necessary package versions that will enable data reduction and not conflict with other packages. | ||
|
|
||
| - Activate the desired environment. After running this command, the selected environment name should appear in parentheses in the command prompt. | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| conda activate YOUR_ENVIRONMENT_NAME | ||
|
|
||
| If you do not remember your environment name, you can run ``conda env list`` to display a list of environments that currently exist. If there is an environment you want to delete, first ensure it is not active, and then run ``conda remove -n YOUR_ENVIRONMENT_NAME --all``. The flat ``--all`` removes the entire environment. | ||
|
|
||
| - Run the following to add the environment to your Jupyter notebook selection. The display name and environment name do not have to be the same. | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| python -m ipykernel install --user --name YOUR_ENVIRONMENT_NAME --display-name YOUR_ENVIRONMENT_NAME | ||
|
|
||
| Install packages and dependencies | ||
| --------------------------------- | ||
|
|
||
| Install Python | ||
| ~~~~~~~~~~~~~~ | ||
|
|
||
| Check the Python version. Use version 3.11 or lower for PyHyperScattering to work. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. see comment in thread -- is this true? it would be better to have a softer recommendation like "use a recent Python version but perhaps not the newest one"
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As of 0.2.9 we are now verified to be compatible with all Python up to current 3.13, so this is incorrect.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm still running into issues. Below is a screenshot for the error I encountered with Python 3.13.2, and I ran into a similar error while when I downgraded to Python 3.13. I meant to attach the full log, but when I enter the file path into my Windows browser, I get a "Windows can't find" error. I don't encounter this error when I use Python 3.12. Noting this for reference until I can troubleshoot further, and then I'll update the instructions.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Those error messages look like you don't have a C compiler to build a 1.x numpy against Python 3.13; idk why it wouldn't be able to find it assuming you do have one. The CI build chain does fail on 3.13/windows but due to the container running out of resources while compiling, so I figured it would work. It does test successfully against 3.13 on Mac and linux. |
||
|
|
||
| .. code-block:: bash | ||
|
|
||
| python --version | ||
|
|
||
| If needed, install the correct Python version. | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| conda install python==3.11 | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is really bad, it will default match to python==3.11.0 which doesn't have a bunch of security and bug fixes. instead, use something like "python<3.12" so you match to 3.11.11 or whatever the newest is. |
||
|
|
||
| If a CondaSSL error is encountered during this step, the following solution can be run, and then Python installation can be retried: https://github.com/conda/conda/issues/8273 | ||
|
|
||
| Install PyHyperScattering | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| ``pip install pyhyperscattering[bluesky,ui]`` installs PyHyperScattering. The ``bluesky`` portion installs Bluesky-related dependencies needed to access the NSLS II Tiled database. The ``ui`` portion installs the necessary dependencies to draw a mask. In some cases, it may be necessary to clone and check out a later PyHyperScattering commit or branch instead of the default version. Below are some examples. | ||
|
|
||
| - ``pip install "git+https://github.com/usnistgov/PyHyperScattering.git#egg=PyHyperScattering[bluesky, ui]"`` installs the latest commit on the main branch. | ||
|
|
||
| - ``pip install "git+https://github.com/usnistgov/PyHyperScattering.git@Issue170_UpdateDatabrokerImports#egg=PyHyperScattering[bluesky, ui]"`` installs the latest commit on the branch named ``Issue170_UpdateDatabrokerImports``. | ||
|
|
||
| - ``pip install "git+https://github.com/usnistgov/PyHyperScattering.git@6657973#egg=PyHyperScattering[bluesky, ui]"`` installs commit ``6657973``. | ||
|
|
||
|
|
||
| Install JupyterLab | ||
| ~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| ``pip install jupyterlab`` is required if using the Anaconda distribution but might not be necessary in other cases (e.g., NSLS II JupyterHub, Google Colab) | ||
|
|
||
| Other installs | ||
| ~~~~~~~~~~~~~~ | ||
|
|
||
| If there are errors during installation or later on, it might be necessary to install additional packages and then retry the pip installs. Below is a list of what might be needed. | ||
|
|
||
| - Microsoft C++ Build Tools (https://visualstudio.microsoft.com/visual-cpp-build-tools/). This is installed outside the Anaconda prompt. Computer should be restarted after this installation. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should explain that some dependencies of PHS use compiled C code and if there is no binary for your system, you need a C compiler such as MS Visual C++, as indicated in the error message you get. |
||
|
|
||
| - ``pip install --upgrade holoviews`` This may be necessary if mask drawing is not working. The ``--upgrade`` is necessary to ensure that the package will get upgraded even if some version of it is currently installed. | ||
|
|
||
| - ``pip install natsort`` allows use of the natsort package, but is not necessary for the main functioning of PyHyperScattering. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ...do we use natsort anywhere? it should be added to requirements if so. if not, it isn't really in scope here? |
||
|
|
||
|
|
||
| Open JupyterLab | ||
| --------------- | ||
|
|
||
| - Start up JupyterLab from the Anaconda command prompt. Do not open JupyterLab using Anaconda's GUI menu. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm confused as to why. |
||
|
|
||
| .. code-block:: bash | ||
|
|
||
| jupyter-lab | ||
|
|
||
| - When prompted to select a kernel, choose the desired environment. If not prompted, ensure that the kernel on the top right-hand corner of the page is set to the correct environment name. | ||
|
|
||
| - Proceed to using a Jupyter notebook of choice to reduce and analyze data. | ||
|
|
||
|
|
||
| Additional resources | ||
| -------------------- | ||
| - Full list of PyHyperScattering dependencies: https://github.com/usnistgov/PyHyperScattering/blob/main/pyproject.toml | ||
| - Further guidance on creating and managing environments: https://jupyter.nsls2.bnl.gov/hub/guide | ||
| - Conda documentation: https://docs.conda.io/projects/conda/en/stable/ | ||
| - Xarray documentation: https://docs.xarray.dev/en/stable/ | ||
| - Numpy documentation: https://numpy.org/doc/2.1/ | ||
| - MatPlotLib documentation: https://matplotlib.org/stable/index.html | ||
|
|
||

There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As GitHub warning says, this needs to be included in the top level toctree.