From e353936c3826f84de1b3eb816170d98813286803 Mon Sep 17 00:00:00 2001 From: Filip Georgievski Date: Sat, 14 Feb 2026 18:01:16 -0500 Subject: [PATCH 1/4] Fix accessibility: add text under headings for MATLAB Statistics notes --- content/notes/matlab-statistics/index.md | 44 +++++++++++++++--------- 1 file changed, 27 insertions(+), 17 deletions(-) diff --git a/content/notes/matlab-statistics/index.md b/content/notes/matlab-statistics/index.md index 64a64301..392e53e7 100644 --- a/content/notes/matlab-statistics/index.md +++ b/content/notes/matlab-statistics/index.md @@ -19,12 +19,13 @@ MATLAB is an integrated technical computing environment from the MathWorks that ## Course Overview - +

This video from the MATLAB academy provides ample introductory information to follow along on this tutorial.

**Video: Statistical Methods with Matlab** ## Exploring Data - +

How do we know what our data looks like? This section aims to show you how to explore your data and get to know what kind of information you are dealing with.

### Visualizing Data Sets +

The following section displays appropriate uses of histograms, boxplots, and scatter plots as a way to quantitatively assess your data before you continue your analysis in MATLAB.

{{< figure src="exploreData1.png" >}} {{< figure src="exploreData2.png" >}} @@ -33,6 +34,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that
### Measures of Centrality and Spread +

This section explores statistical measures such as mean, median, mode, variance, and interquartile range to summarize data.

{{< figure src="centrality1.png" >}} {{< figure src="centrality2.png" >}} {{< figure src="centrality3.png" >}} @@ -55,6 +57,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that ### Distributions +

This section covers different types probability distributions and how to visualize and analyze them in MATLAB.

{{< figure src="distribution3.png" >}} {{< figure src="spread4.png" >}} @@ -65,6 +68,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that **Documentation:   normpdf **     ** unifpdf **     ** randn **     ** rand ** ### Summary +

These figures summarize the key concepts and visualizations discussed in the previous sections.

{{< figure src="summary1.png" >}} @@ -83,6 +87,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that ## Fitting a Curve to Data ### Linear Regression +

This section demonstrates how to fit linear models to data and interpret the results.

{{< figure src="regression1.png" >}} {{< figure src="regression2.png" >}} {{< figure src="regression4.png" >}} @@ -93,6 +98,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that
### Evaluating Goodness of Fit +

Here we evaluate how well linear models fit the data using various statistical metrics, including residual analysis.

{{< figure src="regression5.png" >}} {{< figure src="regression6.png" >}} {{< figure src="regression7.png" >}} @@ -100,18 +106,21 @@ MATLAB is an integrated technical computing environment from the MathWorks that
### Nonlinear Regression +

This section shows how to fit nonlinear models to data and compare them to linear models.

{{< figure src="regression8.png" >}} {{< figure src="regression9.png" >}} {{< figure src="regression10.png" >}} ### Summary +

These figures summarize the results and insights gained from regression analyses and how to fit a curve to data.

{{< figure src="summary4.png" >}} {{< figure src="summary5.png" >}} {{< figure src="summary6.png" >}} ## Interpolating Data - +

This section disucusses how we can create new data as an estimate based off our current data, and implement it in MATLAB

### Linear Interpolation +

Linear interpolation techniques are used to estimate values between known data points.

{{< figure src="interp1.png" >}} {{< figure src="interp2.png" >}} {{< figure src="interp3.png" >}} @@ -120,7 +129,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that {{< figure src="interp6.png" >}} ### Nonlinear Interpolation - +

This section demonstrates nonlinear interpolation methods for more complex datasets.

{{< figure src="interp7.png" >}} {{< figure src="interp8.png" >}} {{< figure src="interp9.png" >}} @@ -130,11 +139,12 @@ MATLAB is an integrated technical computing environment from the MathWorks that
### Summary +

Figures here summarize the interpolation methods and results and how they can be applied using MATLAB.

{{< figure src="summary7.png" >}} {{< figure src="summary8.png" >}} ## Additional Resources - +

This section provides links to additional MATLAB tools, tutorials, and support resources.

{{< figure src="additionalRes1.png" >}}
@@ -143,58 +153,58 @@ MATLAB is an integrated technical computing environment from the MathWorks that
## Exercises - +

The resource computing team has kindly accumulated exercises to practice on, based on your MATLAB needs. All exercises are provided through the MATLAB Help Center

### Visualizing Data sets - +

Practice exercises to reinforce techniques for visualizing height and weight data.

**Exercise: Visualize Height and Weight Data** ### Measure of Centrality and Spread - +

Exercises to calculate and interpret mean, median, standard deviation, and other centrality measures.

**Exercise: Find the Mean and Median** **Exercise: Find the Standard Deviation and IQR** ### Distributions - +

Exercises to explore probability distributions and generate data using MATLAB functions.

**Exercise: Fit and Plot a Normal Distribution** **Exercise: Generating Random Numbers** ### Review: Exploring Data - +

Exercises reviewing data visualization and analysis skills from previous sections.

**Exercise: Earthquakes** ### Linear Regression - +

Exercises to practice fitting lines and polynomials to datasets.

**Exercise: Fit a Line to Data** **Exercise: Fit a Polynomial to Data** ### Evaluating the Goodness of Fit - +

Exercises to evaluate and improve the fit of models to your data.

**Exercise: Evaluate and Improve the Fit**

### Nonlinear Regression - +

Exercises focused on fitting nonlinear models to data and interpreting results.

**Exercise: Fit a Nonlinear Model** ### Review: Fitting a Curve to Data - +

Exercises reviewing linear and nonlinear regression techniques applied to sample datasets.

**Exercise: Temperature Fluctuations** ### Linear Interpolation - +

Exercises to practice estimating values using linear interpolation.

**Exercise: Fill in Missing Data** **Exercise: Resample Data** ### Nonlinear Interpolation - +

Exercises applying nonlinear interpolation methods to datasets.

**Exercise: Resample Data with Different Interpolation Methods** ### Review: Interpolation - +

Exercises reviewing both linear and nonlinear interpolation techniques.

**Exercise: Stock Prices** From 25c7d29c08be13453271790df4f9f772123e9a2e Mon Sep 17 00:00:00 2001 From: Filip Georgievski Date: Sat, 21 Feb 2026 16:43:33 -0500 Subject: [PATCH 2/4] Update matlab statistics index and parallel programming out notes to add missing content under headers --- .../notes/matlab-parallel-programming/out.md | 99 +++++++++++-------- content/notes/matlab-statistics/index.md | 45 ++++++--- 2 files changed, 88 insertions(+), 56 deletions(-) diff --git a/content/notes/matlab-parallel-programming/out.md b/content/notes/matlab-parallel-programming/out.md index 9bec8540..ac8184dd 100644 --- a/content/notes/matlab-parallel-programming/out.md +++ b/content/notes/matlab-parallel-programming/out.md @@ -6,6 +6,8 @@ By Ed Hall # Why parallel computing? +**Parallel computing** can: + * Save time and tackle increasingly complex problems * Reduce computation time by using available compute cores and GPUs * Why parallel computing with MATLAB and Simulink? @@ -17,12 +19,13 @@ By Ed Hall --- -What's the need for Parallel Computing you might ask or what's the motivation for it? +**What's the need for Parallel Computing you might ask or what's the motivation for it?** The size of the problems we need to solve is increasing and there's a growing need to gain faster helping bring products to market quickly. And this need exists for engineers and researchers across multiple industries and applications! Another motivation is simply the fact that hardware is becoming powerful - modern computers, even laptops, are parallel in architecture with multiple processors/cores. Access to GPUs, cluster of computers or even cloud computing infrastructure is becoming common. -However the challenge lies in the fact that you need software to utilize the power of this hardware and hence parallel computing expertise is a requirement -How do MathWorks Parallel Computing Tools help ? +However, the challenge lies in the fact that you need software to utilize the power of this hardware and hence parallel computing expertise is a requirement + +**How do MathWorks Parallel Computing Tools help ?** Enables Engineers, scientists and researchers like yourself leverage the computational power of available hardware without the need to be a parallel computing expert Accelerate your workflows with minimal changes to your existing code Allow you to seamlessly scale your applications and models from your Desktop to clusters in your organization or on the cloud if you need access to more computational power/memory. @@ -45,24 +48,25 @@ Our parallel computing tools are all about enabling you to scale to more hardwar * 3-4 months of development time saved * Validation time sped up 2X -![](img/Matlab-Parallel-ProgrammingFall23_new2.png) +![Automotive Test Analysis](../img/automotive-test-analysis.png) + **Heart Transplant Studies** * 4 weeks reduced to 5 days * Process time sped up 6X -![](img/Matlab-Parallel-ProgrammingFall23_new3.png) +![Heart Transplant Studies](../img/heart-transplant-studies.png) **Discrete-Event Model of Fleet Performance** * Simulation time reduced from months to hours * Simulation time sped up 20X -![](img/Matlab-Parallel-ProgrammingFall23_new4.jpg) +![Discrete-Event Model of Fleet Performance](../img/discrete-model-fleet-performance.png) **Calculating Derived Market Data** * Implementation time reduced by months * Updates sped up 8X - +![Derived Market Data](../img/derived-market-data.jpg) # Accelerating and Parallelizing MATLAB Code * Optimize your serial code for performance @@ -130,7 +134,7 @@ In general you should not run more MATLAB computational engines than the number Note that Parallel Computing Toolbox provides licensing for enough workers to cover all of the physical cores in a single machine. -See also: https://www.mathworks.com/discovery/matlab-multicore.html +See also: [MATLAB Multithreading](https://www.mathworks.com/discovery/matlab-multicore.html) ## Accelerating MATLAB and Simulink Applications @@ -146,8 +150,9 @@ Let's focus first on built in support that can be enabled simply in MATLAB toolb ## Demo: Classification Learner AppAnalyze sensor data for human activity classification +![Classification Demo](../img/classification-demo.png) +![Cellphone Optimization Demo](../img/cellphone-optimization-demo.png) -![](img/Matlab-Parallel-ProgrammingFall23_new7.png) * Objective: visualize and classify cellphone sensor data of human activity * Approach: @@ -156,13 +161,14 @@ Let's focus first on built in support that can be enabled simply in MATLAB toolb --- -https://insidelabs-git.mathworks.com/ltc-ae/demos/HumanActivityRecognition +Open this [link.](https://github.com/sushant1827/Human-Activity-Recognition-with-Smartphones) + Let's open MATLAB and try out a demonstration of using built in parallel functionality invoked by setting a toggle to invoke parallel. In this demo, we are loading IOT sensor data from a mobile phone and then using this sensor data to classify the data into activity-standing, sitting, running, etc. The demo will utilize the multiple cores in my parallel pool to train multiple classifiers simultaneously. -## Demo: Cell Phone Tower OptimizationUsing Parallel-Enabled Functions +## Demo: Cell Phone Tower Optimization Using Parallel-Enabled Functions * Parallel-enabled functions in Optimization Toolbox * Set flags to run optimization in parallel @@ -198,7 +204,7 @@ Deep Learning, Neural Network training and simulation --- -**Signal Processing and Communications ** +**Signal Processing and Communications** GPU-enabled FFT filtering, cross correlation, BER simulations @@ -238,7 +244,7 @@ Examples: parameter sweeps, Monte Carlo simulations No dependencies or communications between tasks -![](img/Matlab-Parallel-ProgrammingFall23_new16.png) +![Explicit Parallelism Diagram](../img/explicit-parallelism-diagram.png) --- @@ -273,7 +279,7 @@ parfor i = 1:5 end ``` -![](img/Matlab-Parallel-ProgrammingFall23_new17.png) +![Explicit Parallelism](../img/explicit-parallelism.png) --- @@ -308,7 +314,7 @@ Spmd, arrayfun, CUDAKernel,labsend, ## Take Advantage of Cluster Hardware -![](img/Matlab-Parallel-ProgrammingFall23_new19.png) +![Cluster Scaling Example](../img/cluster-scaling-example.png) * Offload computation: * Free up desktop @@ -360,7 +366,7 @@ Just submit your jobs and they will run in the background, you can then access r ![](img/Matlab-Parallel-ProgrammingFall23_new22.png) -## Why parallel computing mattersScaling with a compute cluster +## Why parallel computing matters Scaling with a compute cluster ![](img/Matlab-Parallel-ProgrammingFall23_new23.png) @@ -369,7 +375,9 @@ Just submit your jobs and they will run in the background, you can then access r --- In this example, you see a parameter sweep in which we run up to 160,000 different configurations -Doc example https://www.mathworks.com/help/distcomp/examples/plot-progress-during-parallel-computations-using-parfor-and-dataqueue.html?searchHighlight=dataqueue&s_tid=doc_srchtitle + +[Click on this Example](https://www.mathworks.com/help/parallel-computing/plot-progress-during-parallel-computations-using-parfor-and-dataqueue.html) + If my problem is well-sized, I can get more than a 90x speed-up with 100 workers. Just adding more workers is not a guarantee of more speed-up, though. Every application has its limits, and eventually overhead will dominate. @@ -433,6 +441,8 @@ Change hardware without changing algorithm ## Broad Range of Needs and Cloud Environments Supported +Parallel MATLAB workflows can run on desktops, on-premise clusters, or cloud environments with GPU support. Users can choose between pre-configured cloud templates, custom installations, or MathWorks Cloud services depending on resource needs and accessibility requirements. + ![](img/Matlab-Parallel-ProgrammingFall23_new28.png) | Access requirements | Desktop in the cloud | Cluster in the cloud
(Client can be any cloud on on-premise desktop) | @@ -460,7 +470,7 @@ Again, there are a range of environments and ways you can access MATLAB Parallel Note: NVIDIA GPU Cloud is actually a container, and can be run in the cloud or on-premise. -For MATLAB Parallel Server, see: https://www.mathworks.com/products/matlab-parallel-server/get-started.html +For MATLAB Parallel Server, see this [page.](https://www.mathworks.com/products/matlab-parallel-server/get-started.html) # Tackling data-intensive problems on desktops and clusters @@ -499,7 +509,7 @@ Parallel Computing Toolbox extends the tall array and MapReduce capabilities bui * Linear Algebra and Signal Processing * A large number of standard MATLAB functions work with distributed arrays just as they do for normal variables -![](img/Matlab-Parallel-ProgrammingFall23_new30.png) +![Distributed Arrays](../img/distributed-arrays.png) --- @@ -512,7 +522,7 @@ Just like we do for the GPU, we have overloaded functions that work transparentl ## Tall Arrays -![](img/Matlab-Parallel-ProgrammingFall23_new31.png) +![Tall Arrays](../img/tall-arrays-diagram.png) * Applicable when: * Data is **columnar** - with **many** rows @@ -521,8 +531,7 @@ Just like we do for the GPU, we have overloaded functions that work transparentl * Statistical and machine learning applications * Hundreds of functions supported in MATLAB andStatistics and Machine Learning Toolbox -![](img/Matlab-Parallel-ProgrammingFall23_new32.png) - +![Tall Data](../img/tall-data.png) Automatically breaks data up into small “chunks” that fit in memory Tall arrays scan through the dataset one “chunk” at a time @@ -547,7 +556,7 @@ In this example we started with CSV files, which contain tabular data. The resul * Fit linear model * Predict fare and validate model -![](img/Matlab-Parallel-ProgrammingFall23_new33.png) +![Taxi Example](../img/taxi-example.png) # Accelerating applications with NVIDIA GPUs @@ -573,7 +582,7 @@ Dedicated high speed memory Parallel Computing Toolbox requires NVIDIA GPUs -[nvidia.com/object/cuda_gpus.html](http://www.nvidia.com/object/cuda_gpus.html) +[CUDA GPU Compute Capability](http://www.nvidia.com/object/cuda_gpus.html) | MATLAB Release | Required Compute Capability | | :-: | :-: | @@ -581,7 +590,8 @@ Parallel Computing Toolbox requires NVIDIA GPUs | MATLAB R2014b - MATLAB R2017b | 2.0 or greater | | MATLAB R2014a and earlier releases | 1.3 or greater | -## GPU Computing ParadigmNVIDIA CUDA-enabled GPUs +## GPU Computing Paradigm NVIDIA CUDA-enabled GPUs +NVIDIA CUDA-enabled GPUs allow MATLAB to execute thousands of computations simultaneously, providing massive parallelism. MATLAB uses GPU arrays and built-in GPU-enabled functions to offload computationally intensive tasks from the CPU to the GPU for faster performance. ![](img/Matlab-Parallel-ProgrammingFall23_new35.png) @@ -600,7 +610,7 @@ Expected speed-up varies with problem specifics as well as the avaialable hardwa | | Moderate ease of use and moderate control: Common programming constructs(gpuArray, gather)
| | | | Greatest control: Advanced programming constructs | | -## Demo: Wave EquationAccelerating scientific computing in MATLAB with GPUs +## Demo: Wave Equation Accelerating scientific computing in MATLAB with GPUs * Objective: Solve 2nd order wave equation with spectral methods * Approach: @@ -608,13 +618,13 @@ Expected speed-up varies with problem specifics as well as the avaialable hardwa * Modify the code to use GPUcomputing using gpuArray * Compare performance ofthe code using CPU and GPU -![](img/Matlab-Parallel-ProgrammingFall23_new36.png) +![Second Order Wave Equation Example](../img/second-order-wave-equation-example.png) --- ## Speed-up using NVIDIA GPUs -![](img/Matlab-Parallel-ProgrammingFall23_new37.png) +![Speed-up GPU](../img/speed-up-gpu.png) * Ideal Problems * Massively Parallel and/or Vectorized operations @@ -638,6 +648,8 @@ The diagram pretty much sums up the easiest way to do GPU computing in MATLAB - ## GPU Computing with Matlab +MATLAB provides seamless integration with NVIDIA GPUs, enabling users to accelerate computations without needing to write CUDA code. Users can transfer data to the GPU with gpuArray, perform computations, and gather results back to the CPU with minimal changes to their existing code. + **Useful Links** **[GPU Computing](https://www.mathworks.com/help/parallel-computing/gpu-computing.html?s_tid=CRUX_lftnav)** @@ -679,13 +691,14 @@ I've also shown you the specific steps needed. MathWorks provides tools and in * [MATLAB Advanced Software Development Performance and Memory](http://www.mathworks.com/help/matlab/performance-and-memory.html) * [Parallel Computing Toolbox](http://www.mathworks.com/help/distcomp/index.html) * Parallel and GPU Computing Tutorials - * [https://www.mathworks.com/videos/series/parallel-and-gpu-computing-tutorials-97719.html](https://www.mathworks.com/videos/series/parallel-and-gpu-computing-tutorials-97719.html) + * [Parallel Computing Toolbox](https://www.mathworks.com/products/parallel-computing.html) + * [GPU Computing](https://www.mathworks.com/help/parallel-computing/gpu-computing.html) * Parallel Computing on the Cloud with MATLAB - * [http://www.mathworks.com/products/parallel-computing/parallel-computing-on-the-cloud/](http://www.mathworks.com/products/parallel-computing/parallel-computing-on-the-cloud/) + * [Parallel Computing](http://www.mathworks.com/products/parallel-computing/parallel-computing-on-the-cloud/) ## MATLAB Central Community -Every month, over **2 million ** MATLAB & Simulink users visit MATLAB Central to get questions answered, download code and improve programming skills. +Every month, over **2 million** MATLAB & Simulink users visit MATLAB Central to get questions answered, download code and improve programming skills. ![](img/Matlab-Parallel-ProgrammingFall23_new38.png) @@ -697,13 +710,13 @@ Every month, over **2 million ** MATLAB & Simulink users visit MATLAB Central t **[Blogs](http://blogs.mathworks.com/)** : Get the inside view from Engineers who build and support MATLAB & Simulink -**[ThingSpeak](https://thingspeak.com/)** : ** ** Explore IoT Data +**[ThingSpeak](https://thingspeak.com/)** : **Explore** IoT Data And more for you to explore… ## Get Help -![](img/Matlab-Parallel-ProgrammingFall23_new39.png) +![Get Help Example](../img/get-help-example.png) Quick access to: @@ -716,11 +729,11 @@ Support engineers ## Part II: Matlab Parallel Computing On Rivanna * Upload the file Rivanna.zip to your Rivanna account using any of the methods outlined in the following URL. -* [https://www.rc.virginia.edu/userinfo/data-transfer/#data-transfer-methods](https://www.rc.virginia.edu/userinfo/data-transfer/#data-transfer-methods) +* [Data Transfer Methods](https://www.rc.virginia.edu/userinfo/data-transfer/#data-transfer-methods) * Log into Rivanna using the FastX web interface and launch the Mate Desktop. -* [http://rivanna-desktop.hpc.virginia.edu](http://rivanna-desktop.hpc.virginia.edu/) +* [Rivanna Login](https://login5.hpc.virginia.edu:8000/auth/ssh/) * If you are off-grounds, you will have to run the UVA Anywhere VPN before logging in through FastX. -* [https://](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[www.rc.virginia.edu](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[/](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[userinfo](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[/](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[linux](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[/](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[uva](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[-anywhere-](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[vpn](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[-](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[linux](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/)[/](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/) +* [UVA Anywhere VPN on Linux](https://www.rc.virginia.edu/userinfo/linux/uva-anywhere-vpn-linux/) * In the Mate Desktop, open two terminal windows, one to start the Matlab Desktop and one to work from the Rivanna command line. * In the second terminal window, go to the location where you uploaded the Rivanna.zip file and unzip it with the command 'unzip Rivanna.zip' to create a folder Rivanna. @@ -733,14 +746,14 @@ Support engineers * sbatch example1.slurm * Check that the job is in the queue with the command * squeue -u -* [https://www.rc.virginia.edu/userinfo/rivanna/slurm/#displaying-job-status](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#displaying-job-status) +* [Displaying Job Status](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#displaying-job-status) * Once the job is running, it should finish in a few minutes. The error file (.err) should not contain anything and the output file (.out) should contain what Matlab would normally send to the Matlab command window. * If for some reason you need to cancel your job, use the scancel command with the job id -* [https://](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[www.rc.virginia.edu](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[/](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[userinfo](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[/](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[rivanna](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[/](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[slurm](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job)[/#canceling-a-job](https://www.rc.virginia.edu/userinfo/rivanna/slurm/#canceling-a-job) +* [Cancelling a Job](https://www.rc.virginia.edu/userinfo/hpc/slurm/#canceling-a-job) * For the examples of running Matlab using multiple compute nodes, those jobs are submitted from within Matlab rather than with an external slurm script. Matlab creates its own slurm script to submit the job. * The reference for this material is at the URL -* [https://www.rc.virginia.edu/userinfo/rivanna/software/matlab/](https://www.rc.virginia.edu/userinfo/rivanna/software/matlab/) +* [Matlab and UVA HPC](https://www.rc.virginia.edu/userinfo/rivanna/software/matlab/) Using multiple cores on one node: @@ -779,7 +792,7 @@ matlab -nodisplay -r \ ## SLURM Script end-of-job email -![](img/Matlab-Parallel-ProgrammingFall23_new40.png) +![Slurm Email Example](../img/slurm-email-example.png) * When the job finishes, the end-of-job email sent by SLURM will contain the output of the SLURM seff command. @@ -819,7 +832,7 @@ As artificial intelligence (AI) and machine learning (ML) continue to change how * The job (can be single- or multi-GPU) is I/O intensive. * The job (can be single- or multi-GPU) requires more than 40 GB GPU memory. (The non-POD nodes with the highest GPU memory are the regular A100 nodes with 40 GB GPU memory.) -[https://](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[www.rc.virginia.edu](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[/](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[userinfo](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[/](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[rivanna](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[/](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[basepod](https://www.rc.virginia.edu/userinfo/rivanna/basepod/)[/](https://www.rc.virginia.edu/userinfo/rivanna/basepod/) +[HGX 200 Nodes and the NVIDIA DGX BasePOD](https://www.rc.virginia.edu/userinfo/hpc/basepod/) Slurm script additional constraint @@ -868,7 +881,7 @@ Shell script to monitor and record cpu/memory usage using top Shell script to monitor job resource usage output file -![](img/Matlab-Parallel-ProgrammingFall23_new41.png) +![Slurm Output File](../img/slurm-output-file.png) ## Parallel/GPU Computing @@ -886,7 +899,7 @@ The only way to definitively answer this question is to perform a scaling study Plotting the same data on log axes gives a lot more insight. Note the different scales for the left axes on the two plots. Including a line showing linear scaling and plotting the parallel efficiency on the right axis adds even more value -![](img/Matlab-Parallel-ProgrammingFall23_new42.jpg) +![Scaling Log Graphs](../img/scaling-logs-graphs.jpg) ## Where should I be on the scaling curve? diff --git a/content/notes/matlab-statistics/index.md b/content/notes/matlab-statistics/index.md index 392e53e7..4670cb61 100644 --- a/content/notes/matlab-statistics/index.md +++ b/content/notes/matlab-statistics/index.md @@ -9,7 +9,7 @@ toc : true MATLAB is an integrated technical computing environment from the MathWorks that combines array-based numeric computation, advanced graphics and visualization, and a high-level programming language. Separately licensed toolboxes provide additional domain-specific functionality. -**Matlab Academy: Statistical Methods with Matlab** +**Matlab Academy: Statistics Onramp** **Documentation: Statistics and Machine Learning Toolbox (help page)** @@ -19,11 +19,14 @@ MATLAB is an integrated technical computing environment from the MathWorks that ## Course Overview +

This video from the MATLAB academy provides ample introductory information to follow along on this tutorial.

-**Video: Statistical Methods with Matlab** + +**Video: Statistical Methods with Matlab** ## Exploring Data

How do we know what our data looks like? This section aims to show you how to explore your data and get to know what kind of information you are dealing with.

+ ### Visualizing Data Sets

The following section displays appropriate uses of histograms, boxplots, and scatter plots as a way to quantitatively assess your data before you continue your analysis in MATLAB.

{{< figure src="exploreData1.png" >}} @@ -33,6 +36,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that **Documentation:   histogram **     ** boxplot **     ** scatter **
+ ### Measures of Centrality and Spread

This section explores statistical measures such as mean, median, mode, variance, and interquartile range to summarize data.

{{< figure src="centrality1.png" >}} @@ -119,6 +123,7 @@ MATLAB is an integrated technical computing environment from the MathWorks that ## Interpolating Data

This section disucusses how we can create new data as an estimate based off our current data, and implement it in MATLAB

+ ### Linear Interpolation

Linear interpolation techniques are used to estimate values between known data points.

{{< figure src="interp1.png" >}} @@ -148,63 +153,77 @@ MATLAB is an integrated technical computing environment from the MathWorks that {{< figure src="additionalRes1.png" >}}
+ **MATLAB Central**     ** MathWorks Support **
## Exercises

The resource computing team has kindly accumulated exercises to practice on, based on your MATLAB needs. All exercises are provided through the MATLAB Help Center

+ ### Visualizing Data sets

Practice exercises to reinforce techniques for visualizing height and weight data.

-**Exercise: Visualize Height and Weight Data** + +**Exercise: Visualize Height and Weight Data** ### Measure of Centrality and Spread

Exercises to calculate and interpret mean, median, standard deviation, and other centrality measures.

-**Exercise: Find the Mean and Median** -**Exercise: Find the Standard Deviation and IQR** +**Exercise: Find the Mean and Median** + +**Exercise: Find the Standard Deviation and IQR** ### Distributions

Exercises to explore probability distributions and generate data using MATLAB functions.

-**Exercise: Fit and Plot a Normal Distribution** -**Exercise: Generating Random Numbers** +**Exercise: Fit and Plot a Normal Distribution** + + **Exercise: Generating Random Numbers** ### Review: Exploring Data

Exercises reviewing data visualization and analysis skills from previous sections.

-**Exercise: Earthquakes** + +**Exercise: Earthquakes** ### Linear Regression

Exercises to practice fitting lines and polynomials to datasets.

-**Exercise: Fit a Line to Data** -**Exercise: Fit a Polynomial to Data** +**Exercise: Fit a Line to Data** + +**Exercise: Fit a Polynomial to Data** ### Evaluating the Goodness of Fit

Exercises to evaluate and improve the fit of models to your data.

-**Exercise: Evaluate and Improve the Fit** + +**Exercise: Evaluate and Improve the Fit**

+ ### Nonlinear Regression

Exercises focused on fitting nonlinear models to data and interpreting results.

-**Exercise: Fit a Nonlinear Model** + +**Exercise: Fit a Nonlinear Model** ### Review: Fitting a Curve to Data

Exercises reviewing linear and nonlinear regression techniques applied to sample datasets.

-**Exercise: Temperature Fluctuations** + +**Exercise: Temperature Fluctuations** ### Linear Interpolation

Exercises to practice estimating values using linear interpolation.

+ **Exercise: Fill in Missing Data** **Exercise: Resample Data** ### Nonlinear Interpolation

Exercises applying nonlinear interpolation methods to datasets.

+ **Exercise: Resample Data with Different Interpolation Methods** ### Review: Interpolation

Exercises reviewing both linear and nonlinear interpolation techniques.

+ **Exercise: Stock Prices** From af77b0cc46e7f1161f8cfb7c2146a1354571b6c4 Mon Sep 17 00:00:00 2001 From: Aaditi Rai Date: Mon, 23 Feb 2026 11:45:07 -0500 Subject: [PATCH 3/4] RCWT-1299: Images are missing alt text --- .../fiji-image-processing/fiji-omero/index.md | 14 ++++---- content/courses/opencv/index.md | 34 +++++++++---------- .../distributed_mpi_cart_exercise.md | 2 +- content/notes/containers/_index.md | 2 +- content/notes/containers/overview-purpose.md | 4 +-- content/notes/containers/overview-services.md | 2 +- content/notes/containers/overview-vms.md | 2 +- 7 files changed, 30 insertions(+), 30 deletions(-) diff --git a/content/courses/fiji-image-processing/fiji-omero/index.md b/content/courses/fiji-image-processing/fiji-omero/index.md index 1e112287..b566cc0c 100644 --- a/content/courses/fiji-image-processing/fiji-omero/index.md +++ b/content/courses/fiji-image-processing/fiji-omero/index.md @@ -182,8 +182,8 @@ Before you begin, you need to know the dataset ID that the image should be linke 3. Go to OMERO webclient (http://omero.hpc.virginia.edu) and look for the uploaded image in `Orphaned Images`. - - + ariegated leaf specimen with ruler for scale + Export to OMERO dialog with server and dataset settings
@@ -210,16 +210,16 @@ Before you begin, you need to know the dataset ID that the image should be linke a. Go to `Process` > `Filter` > `Median`. In the popup dialog enter a `Radius` of `3.0` and click `OK`. This will smooth out some of the image's intrinsic noise without degrading the object outlines. - b. Go to `Image` > `Adjust Threshold`. In the popup dialog choose the `Default` thresholding algorithm, uncheck the `Dark Background` box and click `Apply`. The image should have been converted to a binary mask with white objects ona black background. - - + b. Go to `Image` > `Adjust Threshold`. In the popup dialog choose the `Default` thresholding algorithm, uncheck the `Dark Background` box and click `Apply`. The image should have been converted to a binary mask with white objects on a black background. + Fiji Threshold dialog with range 126-255 and Red overlay selected + Blobs image with red threshold overlay highlighting segmented cells c. Go to `Analyze` > `Set Measurements...`. In the popup dialog specify the parameters as shown in this screenshot. Click `OK`. d. Go to `Analyze` > `Analyze Particles` and set up the parameters as shown. Click `OK`. - - + Set Measurements dialog settings + Analyze e. These steps should create a `Results` and a `Summary` table. diff --git a/content/courses/opencv/index.md b/content/courses/opencv/index.md index 494d1179..b91076dc 100644 --- a/content/courses/opencv/index.md +++ b/content/courses/opencv/index.md @@ -51,7 +51,7 @@ If you have a Rivanna account, you can work through this tutorial using an [Open 4. On the next screen, specify resources as shown in this screenshot: - ![](ood-resources.png) + ![Screenshot of the Rivanna Open OnDemand “Desktop” launch page showing fields to select session time, CPU cores, memory, allocation (SUs), partition, and a blue “Launch” button at the bottom.](ood-resources.png) >**Note:** Workshop participants may specify `rivanna-training` in the `Allocation (SUs)` field. Alternatively, you may use any other Rivanna allocation that you are a member of. @@ -169,7 +169,7 @@ cv2.waitKey(0) cv2.destroyAllWindows() ``` -![](clown.png) +![Image of a clown](clown.png) The `cv2.imshow()` method displays the image on our screen. The `cv2.waitKey()` function waits for a key to be pressed. This is important otherwise our image would display and immediately disappear before we even see the image. The call of `destroyAllWindows()` should be placed at the end of any script that uses the `imshow` function. @@ -255,7 +255,7 @@ ax[1].axis('off') -![](clown-resized.png) +![Side-by-side comparison of a distorted resized clown image (left) and a proportionally resized clown image (right).](clown-resized.png) ## Splitting and Merging of Color Channels @@ -283,7 +283,7 @@ axarr[1,1].axis('off') axarr[1,1].set_title('RGB') ``` -![](clown-split.png) +![Four-panel comparison of a clown image with individual red, green, and blue channels in grayscale and the full-color RGB image.](clown-split.png) Let's take the blue and green channel only and merge them back into a new RGB image, effectively masking the red channel. For this we'll define a new numpy array with the same width and height as the original image and a depth of 1 (single channel), all pixels filled with zero values. Since the individual channels of an RGB image are 8-bit numpy arrays, we choose the numpy `uint8` data type. @@ -300,7 +300,7 @@ ax.imshow(cv2.cvtColor(merged, cv2.COLOR_BGR2RGB)) ax.axis('off') ``` -![](clown-merged.png) +![Picture of a clown merged with all the color channels](clown-merged.png) # Exercises @@ -329,7 +329,7 @@ Common arguments are: Let's try this with a noisy version of the clown image. This is a color RGB image and so we'll try the `cv2.fastNlMeansDenoisingColored()` filter. Here is the noisy input image `clown-noisy.png`. -![](clown-noisy.png) +![Noisy version of the original clown image](clown-noisy.png) The `denoising.py` script demonstrates how it works. @@ -355,7 +355,7 @@ plt.subplot(122),plt.imshow(cv2.cvtColor(denoised, cv2.COLOR_BGR2RGB), interpola plt.show() ``` -![](clown-noisy-denoised.png) +![Comparison of the noisy clown image (left) and the denoised clown image (right)](clown-noisy-denoised.png) Additional useful filters for smoothing images: * `GaussianBlur` - blurs an image using a Gaussian filter @@ -401,7 +401,7 @@ ax[2].axis('off') Original | Dilation | Erosion :-------------------:|:----------------------:|:----------------------: -![](morph-input.png) | ![](morph-dilated.png) | ![](morph-eroded.png) | +![original image](morph-input.png) | ![dilated image](morph-dilated.png) | ![eroded image](morph-eroded.png) | > By increasing the kernel size or number of iterations we can dilate or erode more of the original object. @@ -421,7 +421,7 @@ closing = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel) Original | Opening | Closing :-------------------:|:----------------------:|:----------------------: -![](morph-input.png) | ![](morph-opened.png) | ![](morph-closed.png) | +![Original image](morph-input.png) | ![Opened image](morph-opened.png) | ![Closed image](morph-closed.png) | > Note how the opening operation removed the small dot in the top right corner. In contrast the closing operation retained that small object and also filled in the black hole in the object on the left side of the input image. @@ -434,7 +434,7 @@ gradient = cv2.morphologyEx(image, cv2.MORPH_GRADIENT, kernel) Original | Gradient (edges) :-------------------:|:----------------------: -![](morph-input.png) | ![](morph-gradient.png) +![Original image](morph-input.png) | ![Image with just the gradients](morph-gradient.png) # Exercises @@ -457,7 +457,7 @@ Let's try to identify and measure the area of the nuclei in this image with fluo The complete code is in the `segmentation.py` script. -![](fluorescent-cells.png) +![Image with flourescent labeled cells](fluorescent-cells.png) ## Preprocessing @@ -470,7 +470,7 @@ image = cv2.imread('fluorescent-cells.png') nuclei = image[:,:,0] # get blue channel ``` -![](nuclei.png) +![Image of fluorescent cells with the blue channel extracted](nuclei.png) To eliminate noise, we apply a Gaussian filter with 3x3 kernel, then apply the Otsu thresholding algorithm. The thresholding converts the grayscale intensity image into a black and white binary image. The function returns two values, we store them in `ret` (the applied threshold value) and `thresh` (the thresholded black & white binary image). White pixels represent nuclei; black pixel represent background. @@ -496,7 +496,7 @@ opening = clear_border(opening) The resulting image looks like this. -![](nuclei-opening.png) +![Opening operation applied on the image of fluorescent cells](nuclei-opening.png) A tricky issue is that some of the nuclei masks are touching each other. We need to find a way to break up these clumps. We do this in several steps. First, we will dilate the binary nuclei mask. The black areas in the resulting image represent pixels that certainly do not contain any nuclear components. We call it the `sure_bg`. @@ -505,7 +505,7 @@ A tricky issue is that some of the nuclei masks are touching each other. We nee sure_bg = cv2.dilate(opening,kernel,iterations=10) ``` -![](nuclei-sure_bg.png) +![Image of fluorescent cells with a sure background](nuclei-sure_bg.png) The nuclei are all fully contained inside the white pixel area. The next step is to find estimates for the center of each nucleus. Some of the white regions may contain more than one nucleus and we need to separate the joined ones. We calculate the distance transform to do this. @@ -521,7 +521,7 @@ ret, sure_fg = cv2.threshold(dist_transform,0.6*dist_transform.max(),255,0) Distance Transform | Sure Foreground (nuclei seeds) :-------------------------------:|:------------------------: -| ![](nuclei-dist_transform.png) | ![](nuclei-sure_fg.png) | +| ![Distance transform map for the image of the fluorescent cells](nuclei-dist_transform.png) | ![Sure foreground for image](nuclei-sure_fg.png) | By subtracting the sure foreground regions from the sure background regions we can identify the regions of unknown association, i.e. the pixels that we have not assigned to be either nuclei or background. @@ -534,7 +534,7 @@ unknown = cv2.subtract(sure_bg,sure_fg) Sure Background | Sure Foreground (nuclei seeds) | Unknown :----------------------:|:-------------------------------:|:-----------------------: -![](nuclei-sure_bg.png) | ![](nuclei-sure_fg.png) | ![](nuclei-unknown.png) | +![Image of fluorescent cells with a sure background](nuclei-sure_bg.png) | ![Image of fluorescent cells with a sure foreground](nuclei-sure_fg.png) | ![Image of fluorescent cells with an unknown background](nuclei-unknown.png) | ## Watershed @@ -563,7 +563,7 @@ The resulting `markers` (pseudo-colored) and input images with segmentation over Markers | Segmentation :----------------------:|:-------------------------: -![](nuclei-markers.png) | ![](image-segmented.png) | +![Images of nuclei with pseudo-colored markers](nuclei-markers.png) | ![Image with segmentation overlay](image-segmented.png) | ## Measure diff --git a/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md b/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md index 604a6865..53b5129c 100644 --- a/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md +++ b/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md @@ -18,7 +18,7 @@ In the language of your choice, write a heated-plate solution using two-dimensio Check your rsults by plotting. If you wish, you can use the script contour2d.py {{< spoiler >}} -{{< code-download file=/courses/parallel-computing-introduction/scripts/contour2d.py" lang="python" >}} +{{< code-download file="courses/parallel-computing-introduction/scripts/contour2d.py" lang="python" >}} {{< /spoiler >}} The usage is diff --git a/content/notes/containers/_index.md b/content/notes/containers/_index.md index 61b61002..c5fef2bd 100755 --- a/content/notes/containers/_index.md +++ b/content/notes/containers/_index.md @@ -18,4 +18,4 @@ menu: You've developed an app and written the paper. Now it's time to deploy the app so the world (and the reviewers) can see how awesome it is. This is Part 1 of a two-part workshop that will cover how to deploy web apps for publication. In Part 1 we will go over how to containerize our apps with Docker and maintain them with GitHub. - +docker logo img github logo image diff --git a/content/notes/containers/overview-purpose.md b/content/notes/containers/overview-purpose.md index 02ef04e9..b007d7f2 100755 --- a/content/notes/containers/overview-purpose.md +++ b/content/notes/containers/overview-purpose.md @@ -17,5 +17,5 @@ A container is a single unit of software that contains all the packages and code There are several container technologies out there, but the big ones are Docker and Singularity. Docker is what you will encounter most often in the wild. Singularity (now called Apptainer) is used on HPC systems where most users don't have root access. - - \ No newline at end of file +docker logo image +apptainer logo image \ No newline at end of file diff --git a/content/notes/containers/overview-services.md b/content/notes/containers/overview-services.md index bcdb72f4..dc65195f 100755 --- a/content/notes/containers/overview-services.md +++ b/content/notes/containers/overview-services.md @@ -15,6 +15,6 @@ Container-based architecture, also known as "microservices," is an approach to d Containers are portable, efficient, reusable, and contain code and any dependencies in a single package. Containerized services typically run a single process, rather than an entire stack within the same environment. This allows developers to replace, scale, or troubleshoot portions of their entire application at a time. - +Diagram of multiple containers running separate services in a cluster. Research Computing runs microservices in a clustered orchestration environment that automates the deployment and management of many containers easy and scalable. This cluster has >1000 cores and ~1TB of memory allocated to running containerized services. It also has over 300TB of cluster storage and can attach to project and value storage. \ No newline at end of file diff --git a/content/notes/containers/overview-vms.md b/content/notes/containers/overview-vms.md index f185d711..23aa4c74 100755 --- a/content/notes/containers/overview-vms.md +++ b/content/notes/containers/overview-vms.md @@ -15,4 +15,4 @@ You may be familiar with the virtual machines (VMs), which accomplish the same g The size of Docker images is usually on the order of tens of MBs, while VMs can be several tens of GBs large. - \ No newline at end of file +Side-by-side comparison of containerized applications (Docker on a host OS) and virtual machines (apps running on guest OSs over a hypervisor). \ No newline at end of file From 58561382c8060783b6d87c039a119b45d947665b Mon Sep 17 00:00:00 2001 From: Katherine Holcomb Date: Wed, 11 Mar 2026 21:12:34 -0400 Subject: [PATCH 4/4] Fixing problem previously introduced --- .../distributed_mpi_cart_exercise.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md b/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md index 8642ec02..070910cd 100644 --- a/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md +++ b/content/courses/parallel-computing-introduction/distributed_mpi_cart_exercise.md @@ -17,15 +17,10 @@ In the language of your choice, write a heated-plate solution using two-dimensio Check your rsults by plotting. If you wish, you can use the script contour2d.py: -<<<<<<< HEAD {{< spoiler text="contour2d.py source code" >}} {{< code-download file="/courses/parallel-computing-introduction/scripts/contour2d.py" lang="python" >}} -======= -{{< spoiler >}} -{{< code-download file="courses/parallel-computing-introduction/scripts/contour2d.py" lang="python" >}} ->>>>>>> af77b0cc46e7f1161f8cfb7c2146a1354571b6c4 {{< /spoiler >}} -
+ The usage is ```sh python contour2d.py -r numrows -c numcols basefilename