Skip to content

Commit dd9b14b

Browse files
add examples of the two main ways to use the package in README.md
1 parent 3aa47e9 commit dd9b14b

File tree

1 file changed

+94
-0
lines changed

1 file changed

+94
-0
lines changed

README.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,97 @@ JobQueueMPI.jl has the following components:
3333

3434
- `Controller`: The controller is responsible for managing the jobs and the workers. It keeps track of the jobs that have been sent and received and sends the jobs to the available workers.
3535
- `Worker`: The worker is responsible for executing the jobs. It receives the jobs from the controller, executes them, and sends the results back to the controller.
36+
37+
Users can call functions to compute jobs in parallel in two ways:
38+
- Building a function and using a `pmap` implementation that will put the function in the job queue and send it to the workers.
39+
```julia
40+
using JobQueueMPI
41+
42+
function sum_100(value)
43+
return value + 100
44+
end
45+
46+
sum_100_answer = JobQueueMPI.pmap(sum_100, collect(1:10))
47+
```
48+
- Building the jobs and sending them to workers explicitly. There are examples of this structure in the test folder. This way is much more flexible than the first one, but it requires more code and knowledge about how MPI works.
49+
50+
```julia
51+
using JobQueueMPI
52+
53+
mutable struct Message
54+
value::Int
55+
vector_idx::Int
56+
end
57+
58+
all_jobs_done(controller) = JQM.is_job_queue_empty(controller) && !JQM.any_pending_jobs(controller)
59+
60+
function sum_100(message::Message)
61+
message.value += 100
62+
return message
63+
end
64+
65+
function update_data(new_data, message::Message)
66+
idx = message.vector_idx
67+
value = message.value
68+
return new_data[idx] = value
69+
end
70+
71+
function workers_loop()
72+
if JQM.is_worker_process()
73+
worker = JQM.Worker()
74+
while true
75+
job = JQM.receive_job(worker)
76+
message = JQM.get_message(job)
77+
if message == JQM.TerminationMessage()
78+
break
79+
end
80+
return_message = sum_100(message)
81+
JQM.send_job_answer_to_controller(worker, return_message)
82+
end
83+
exit(0)
84+
end
85+
end
86+
87+
function job_queue(data)
88+
JQM.mpi_init()
89+
JQM.mpi_barrier()
90+
91+
T = eltype(data)
92+
N = length(data)
93+
94+
if JQM.is_controller_process()
95+
new_data = Array{T}(undef, N)
96+
97+
controller = JQM.Controller(JQM.num_workers())
98+
99+
for i in eachindex(data)
100+
message = Message(data[i], i)
101+
JQM.add_job_to_queue!(controller, message)
102+
end
103+
104+
while !all_jobs_done(controller)
105+
if !JQM.is_job_queue_empty(controller)
106+
JQM.send_jobs_to_any_available_workers(controller)
107+
end
108+
if JQM.any_pending_jobs(controller)
109+
job_answer = JQM.check_for_job_answers(controller)
110+
if !isnothing(job_answer)
111+
message = JQM.get_message(job_answer)
112+
update_data(new_data, message)
113+
end
114+
end
115+
end
116+
117+
JQM.send_termination_message()
118+
119+
return new_data
120+
end
121+
workers_loop()
122+
JQM.mpi_barrier()
123+
JQM.mpi_finalize()
124+
return nothing
125+
end
126+
127+
data = collect(1:10)
128+
new_data = job_queue(data)
129+
```

0 commit comments

Comments
 (0)