Skip to content

Commit

Permalink
add examples of the two main ways to use the package in README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
guilhermebodin committed May 14, 2024
1 parent 3aa47e9 commit dd9b14b
Showing 1 changed file with 94 additions and 0 deletions.
94 changes: 94 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,97 @@ JobQueueMPI.jl has the following components:

- `Controller`: The controller is responsible for managing the jobs and the workers. It keeps track of the jobs that have been sent and received and sends the jobs to the available workers.
- `Worker`: The worker is responsible for executing the jobs. It receives the jobs from the controller, executes them, and sends the results back to the controller.

Users can call functions to compute jobs in parallel in two ways:
- Building a function and using a `pmap` implementation that will put the function in the job queue and send it to the workers.
```julia
using JobQueueMPI

function sum_100(value)
return value + 100
end

sum_100_answer = JobQueueMPI.pmap(sum_100, collect(1:10))
```
- Building the jobs and sending them to workers explicitly. There are examples of this structure in the test folder. This way is much more flexible than the first one, but it requires more code and knowledge about how MPI works.

```julia
using JobQueueMPI

mutable struct Message
value::Int
vector_idx::Int
end

all_jobs_done(controller) = JQM.is_job_queue_empty(controller) && !JQM.any_pending_jobs(controller)

function sum_100(message::Message)
message.value += 100
return message
end

function update_data(new_data, message::Message)
idx = message.vector_idx
value = message.value
return new_data[idx] = value
end

function workers_loop()
if JQM.is_worker_process()
worker = JQM.Worker()
while true
job = JQM.receive_job(worker)
message = JQM.get_message(job)
if message == JQM.TerminationMessage()
break
end
return_message = sum_100(message)
JQM.send_job_answer_to_controller(worker, return_message)
end
exit(0)
end
end

function job_queue(data)
JQM.mpi_init()
JQM.mpi_barrier()

T = eltype(data)
N = length(data)

if JQM.is_controller_process()
new_data = Array{T}(undef, N)

controller = JQM.Controller(JQM.num_workers())

for i in eachindex(data)
message = Message(data[i], i)
JQM.add_job_to_queue!(controller, message)
end

while !all_jobs_done(controller)
if !JQM.is_job_queue_empty(controller)
JQM.send_jobs_to_any_available_workers(controller)
end
if JQM.any_pending_jobs(controller)
job_answer = JQM.check_for_job_answers(controller)
if !isnothing(job_answer)
message = JQM.get_message(job_answer)
update_data(new_data, message)
end
end
end

JQM.send_termination_message()

return new_data
end
workers_loop()
JQM.mpi_barrier()
JQM.mpi_finalize()
return nothing
end

data = collect(1:10)
new_data = job_queue(data)
```

0 comments on commit dd9b14b

Please sign in to comment.