IPC
A set of primitives that extend Torch for high performance
parallel computation across thread and process boundaries.
Tree
Implements an AllReduce style binary tree of connected processes
on top of Client-Server nodes. This enables AllReduce style operations
on Tensors across a set of machines. The Tree is topology aware
such that it creates an optimal binary tree of processes
where each link is the fastest possible communication means
available. Allows user code to use one abstraction for
parallel Tensor computation and get high performance without
coding specifically for the underlying topology.
-- Every node will end up with the sum of all nodes
tree.allReduce(grads, function(a, b)
return a:add(b)
end)
See the AllReduce example to try it out.
SlurmTree
An implementation of Tree that integrates with the Slurm cluster manager.
It builds the communication tree by reading in the slurm variables which are
specified via SBATCH directives
(i.e. --nodes, --tasks-per-node, etc...) and minimizing the inter node
communication (when there are more than one tasks per node)
SlurmTree takes two optional arguments:
- File path - For the file that coordinates the initial connection
of processes. The file location has to be shared across nodes.
(By default '~/.torch') - Tasks per gpu - Used to calculate the gpu id property (By default 1)
See the slurm script for an example of how to
start the processes.
Client-Server
A classic client-server implementation over TCP sockets, used for IPC.
Can transfer all Lua primitives as well as Torch Tensor and Storage
types across the network. Includes a very fast implementation for
sending CUDA Tensors between machines and an even faster implementation
for passing CUDA Tensors between GPUs on the same machine using
the PCI-E bus via CUDA IPC.
The implementation is not tied to any specific cluster or discovery
mechanism. All you need to do is ensure your nodes can reach each
other over TCP.
-- Create a server
local server = ipc.server('127.0.0.1', 8080)
-- Create a client and connect to the server
local client = ipc.client('127.0.0.1', 8080)
-- Say hello
client:send('hi')
-- Listen for any client to say something
local msg = server:recvAny()
assert(msg == 'hi')
-- Disconnect and shutdown
client:close()
server:close()
Map
A map function to spawn a set of worker threads and run a
computation in the background. This is very handy for doing
IO off of the main thread, as IO is usually blocked on a
file or socket descriptor.
-- See examples/map.lua for the complete listing
-- Load 3 files in parallel
local t1,t2,t3 = ipc.map(3, function(fileNames, mapid)
return torch.load(fileNames[mapid])
end, {'f1.t7', 'f2.t7', 'f3.t7'}):join()
Read more complete documentation on ipc.map.
Workqueue
A simple single writer multiple reader command queue. Really useful when
combined with map to keep a bunch of background threads grabbing work
off the queue, processing it and then returning answers back to the main
thread.
-- See examples/workqueue.lua for the complete listing
-- Create a named workqueue
local q = ipc.workqueue('my queue')
-- Create 2 background workers that read from the named workqueue
local workers = ipc.map(2, function()
-- This function is not a closure, it is a totally clean Lua environment
local ipc = require 'libipc'
-- Open the queue by name (the main thread already created it)
local q = ipc.workqueue('my queue')
repeat
-- Read the next file name off the workqueue
local fileName = q:read()
if fileName then
-- Load the file and write its contents back into the workqueue
q:write(torch.load(fileName))
end
until fileName == nil
end)
-- Write the file names into the workqueue
q:write('f1.t7')
q:write('f2.t7')
q:write('f3.t7')
-- Read back the 3 answers and print them
print(q:read())
print(q:read())
print(q:read())
Read more complete documentation on ipc.workqueue.
Channels
Channels are a thread synchronization primitive based on
message-passing. Threads communicate via channels by writing messages
onto them and reading messages out of them, in FIFO order. There is no
restriction on which threads or how many threads can read or write
from a channel. This allows one to define concurrent workflows easily.
Channels can also be closed, which prevents further writes to it. Once
all items are read from a closed channel, that channel becomes drained
and nothing further can be read from it. DAGs of computation made up
of channels can be shut down via cascading closing/draining of
channels.
The
producer-consumer example
shows a group of producer threads and a group of consumer threads
being set up to communicate via a channel. The main thread tears the
entire setup down by closing the channel.
The
local model parallelism for forward inference example shows
how to set up a nn.Sequential-based model so that each of its
submodules can execute forward inference in parallel.
The full documentation can be found at ipc.channel.
Examples
Simple scripts you can run locally can be found here.
See the unit tests for a ton more detailed examples.
Documentation
Full API documentation can be found here.
License
Licensed under the Apache License, Version 2.0.
See LICENSE file.