GraphBLAS: graph algorithms in the language of linear algebra
GraphBLAS is a library for creating graph algorithms based on sparse linear algebraic operations over semirings. Visit http://graphblas.org for more details and resources. See also the SuiteSparse:GraphBLAS User Guide in this package.
SuiteSparse:GraphBLAS, (c) 2017-2020, Tim Davis, Texas A&M University, http://faculty.cse.tamu.edu/davis
Contents
- GraphBLAS: faster and more general sparse matrices for MATLAB
- Sparse integer matrices
- Sparse single-precision matrices
- Mixing MATLAB and GraphBLAS matrices
- Faster matrix operations
- A wide range of semirings
- The max.plus tropical semiring
- A boolean semiring
- GraphBLAS operators, monoids, and semirings
- Element-wise operations
- Subtracting two matrices
- Element-wise 'multiplication'
- Overloaded operators
- Overloaded functions
- Zeros are handled differently
- Displaying contents of a GraphBLAS matrix
- Storing a matrix by row or by column
- Hypersparse matrices
- numel uses vpa if the matrix is really huge
- The mask and accumulator
- The descriptor
- Integer arithmetic is different in GraphBLAS
- An example graph algorithm: breadth-first search
- Example graph algorithm: Luby's method in GraphBLAS
- Sparse deep neural network
- Solving the sparse deep neural network problem with GraphbLAS
- Solving the sparse deep neural network problem with MATLAB
- For objects, GraphBLAS has better colon notation than MATLAB
- Iterative solvers work as-is
- ... even in single precision
- Extreme performance differences between GraphBLAS and MATLAB.
- Sparse logical indexing is much, much faster in GraphBLAS
- First method in GraphBLAS, with GrB.assign
- Second method in GraphBLAS, with C(M)=A(M)
- Now with MATLAB matrices, with C(M)=A(M)
- Limitations and their future solutions
- GraphBLAS operations
GraphBLAS: faster and more general sparse matrices for MATLAB
GraphBLAS is not only useful for creating graph algorithms; it also supports a wide range of sparse matrix data types and operations. MATLAB can compute C=A*B with just two semirings: 'plus.times.double' and 'plus.times.complex' for complex matrices. GraphBLAS has 1,473 unique built-in semirings, such as 'max.plus' (https://en.wikipedia.org/wiki/Tropical_semiring). These semirings can be used to construct a wide variety of graph algorithms, based on operations on sparse adjacency matrices.
MATLAB and GraphBLAS both provide sparse matrices of type double, logical, and double complex. GraphBLAS adds sparse matrices of type: single, int8, int16, int32, int64, uint8, uint16, uint32, uint64, and single complex (with MATLAB matrices, these types can only be held in full matrices).
clear GrB.clear format compact rng ('default') ; X = 100 * rand (2) ; G = GrB (X) % GraphBLAS copy of a matrix X, same type
G = 2x2 GraphBLAS double matrix, sparse by col: 4 nonzeros, 4 entries (1,1) 81.4724 (2,1) 90.5792 (1,2) 12.6987 (2,2) 91.3376
Sparse integer matrices
Here's an int8 version of the same matrix:
S = int8 (G) % convert G to a full MATLAB int8 matrix G = GrB (X, 'int8') % a GraphBLAS sparse int8 matrix
S = 2×2 int8 matrix 81 13 91 91 G = 2x2 GraphBLAS int8_t matrix, sparse by col: 4 nonzeros, 4 entries (1,1) 81 (2,1) 91 (1,2) 13 (2,2) 91
Sparse single-precision matrices
Matrix operations in GraphBLAS are typically as fast, or faster than MATLAB. Here's an unfair comparison: computing X^2 with MATLAB in double precision and with GraphBLAS in single precision. You would naturally expect GraphBLAS to be faster.
Please wait ...
n = 1e5 ; X = spdiags (rand (n, 201), -100:100, n, n) ; G = GrB (X, 'single') ; tic G2 = G^2 ; gb_time = toc ; tic X2 = X^2 ; matlab_time = toc ; fprintf ('\nGraphBLAS time: %g sec (in single)\n', gb_time) ; fprintf ('MATLAB time: %g sec (in double)\n', matlab_time) ; fprintf ('Speedup of GraphBLAS over MATLAB: %g\n', ... matlab_time / gb_time) ;
GraphBLAS time: 1.48328 sec (in single) MATLAB time: 5.8599 sec (in double) Speedup of GraphBLAS over MATLAB: 3.95062
Mixing MATLAB and GraphBLAS matrices
The error in the last computation is about eps('single') since GraphBLAS did its computation in single precision, while MATLAB used double precision. MATLAB and GraphBLAS matrices can be easily combined, as in X2-G2. The sparse single precision matrices take less memory space.
err = norm (X2 - G2, 1) / norm (X2,1) eps ('single') whos G G2 X X2
err = 1.50487018943138e-07 ans = single 1.192093e-07 Name Size Bytes Class Attributes G 100000x100000 241879764 GrB G2 100000x100000 481518564 GrB X 100000x100000 322238408 double sparse X2 100000x100000 641756808 double sparse
Faster matrix operations
But even with standard double precision sparse matrices, GraphBLAS is typically faster than the built-in MATLAB methods. Here's a fair comparison:
G = GrB (X) ; tic G2 = G^2 ; gb_time = toc ; err = norm (X2 - G2, 1) / norm (X2,1) fprintf ('\nGraphBLAS time: %g sec (in double)\n', gb_time) ; fprintf ('MATLAB time: %g sec (in double)\n', matlab_time) ; fprintf ('Speedup of GraphBLAS over MATLAB: %g\n', ... matlab_time / gb_time) ;
err = 0 GraphBLAS time: 1.80984 sec (in double) MATLAB time: 5.8599 sec (in double) Speedup of GraphBLAS over MATLAB: 3.23781
A wide range of semirings
MATLAB can only compute C=A*B using the standard '+.*.double' and '+.*.complex' semirings. A semiring is defined in terms of a string, 'add.mult.type', where 'add' is a monoid that takes the place of the additive operator, 'mult' is the multiplicative operator, and 'type' is the data type for the two inputs to the mult operator.
In the standard semiring, C=A*B is defined as:
C(i,j) = sum (A(i,:).' .* B(:,j))
using 'plus' as the monoid and 'times' as the multiplicative operator. But in a more general semiring, 'sum' can be any monoid, which is an associative and commutative operator that has an identity value. For example, in the 'max.plus' tropical algebra, C(i,j) for C=A*B is defined as:
C(i,j) = max (A(i,:).' + B(:,j))
This can be computed in GraphBLAS with:
C = GrB.mxm ('max.+', A, B)
n = 3 ; A = rand (n) ; B = rand (n) ; C = zeros (n) ; for i = 1:n for j = 1:n C(i,j) = max (A (i,:).' + B (:,j)) ; end end C2 = GrB.mxm ('max.+', A, B) ; fprintf ('\nerr = norm (C-C2,1) = %g\n', norm (C-C2,1)) ;
err = norm (C-C2,1) = 0
The max.plus tropical semiring
Here are details of the "max.plus" tropical semiring. The identity value is -inf since max(x,-inf) = max (-inf,x) = x for any x. The identity for the conventional "plus.times" semiring is zero, since x+0 = 0+x = x for any x.
GrB.semiringinfo ('max.+.double') ;
GraphBLAS Semiring: max.+.double (built-in) GraphBLAS Monoid: semiring->add (built-in) GraphBLAS BinaryOp: monoid->op (built-in) z=max(x,y) GraphBLAS type: ztype double size: 8 GraphBLAS type: xtype double size: 8 GraphBLAS type: ytype double size: 8 identity: [ -Inf ] terminal: [ Inf ] GraphBLAS BinaryOp: semiring->multiply (built-in) z=plus(x,y) GraphBLAS type: ztype double size: 8 GraphBLAS type: xtype double size: 8 GraphBLAS type: ytype double size: 8
A boolean semiring
MATLAB cannot multiply two logical matrices. MATLAB R2019a converts them to double and uses the conventional +.*.double semiring instead. In GraphBLAS, this is the common Boolean 'or.and.logical' semiring, which is widely used in linear algebraic graph algorithms.
GrB.semiringinfo ('|.&.logical') ;
GraphBLAS Semiring: |.&.logical (built-in) GraphBLAS Monoid: semiring->add (built-in) GraphBLAS BinaryOp: monoid->op (built-in) z=or(x,y) GraphBLAS type: ztype bool size: 1 GraphBLAS type: xtype bool size: 1 GraphBLAS type: ytype bool size: 1 identity: [ 0 ] terminal: [ 1 ] GraphBLAS BinaryOp: semiring->multiply (built-in) z=and(x,y) GraphBLAS type: ztype bool size: 1 GraphBLAS type: xtype bool size: 1 GraphBLAS type: ytype bool size: 1
clear A = sparse (rand (3) > 0.5) B = sparse (rand (3) > 0.2)
A = 3×3 sparse logical array (2,1) 1 (2,2) 1 (3,2) 1 (1,3) 1 B = 3×3 sparse logical array (1,1) 1 (2,1) 1 (3,1) 1 (1,2) 1 (2,2) 1 (3,2) 1 (1,3) 1 (2,3) 1 (3,3) 1
try % MATLAB R2019a does this by casting A and B to double C1 = A*B catch % MATLAB R2018a throws an error fprintf ('MATLAB R2019a required for C=A*B with logical\n') ; fprintf ('matrices. Explicitly converting to double:\n') ; C1 = double (A) * double (B) end C2 = GrB (A) * GrB (B)
C1 = (1,1) 1 (2,1) 2 (3,1) 1 (1,2) 1 (2,2) 2 (3,2) 1 (1,3) 1 (2,3) 2 (3,3) 1 C2 = 3x3 GraphBLAS bool matrix, sparse by col: 9 nonzeros, 9 entries (1,1) 1 (2,1) 1 (3,1) 1 (1,2) 1 (2,2) 1 (3,2) 1 (1,3) 1 (2,3) 1 (3,3) 1
Note that C1 is a MATLAB sparse double matrix, and contains non-binary values. C2 is a GraphBLAS logical matrix.
whos GrB.type (C2)
Name Size Bytes Class Attributes A 3x3 68 logical sparse B 3x3 113 logical sparse C1 3x3 176 double sparse C2 3x3 1071 GrB ans = 'logical'
GraphBLAS operators, monoids, and semirings
The C interface for SuiteSparse:GraphBLAS allows for arbitrary types and operators to be constructed. However, the MATLAB interface to SuiteSparse:GraphBLAS is restricted to pre-defined types and operators: a mere 13 types, 204 unary operators, 385 binary operators, 77 monoids, 16 select operators, and 2,438 semirings (1,473 of which are unique, since some binary operators are equivalent: 'min.logical' and '&.logical' are the same thing, for example).
That gives you a lot of tools to create all kinds of interesting graph algorithms. For example:
GrB.bfs % breadth-first search GrB.dnn % sparse deep neural network (http://graphchallenge.org) GrB.mis % maximal independent set
See 'help GrB.binopinfo' for a list of the binary operators, and 'help GrB.monoidinfo' for the ones that can be used as the additive monoid in a semiring. 'help GrB.unopinfo' lists the unary operators.
help GrB.binopinfo
GRB.BINOPINFO list the details of a GraphBLAS binary operator. GrB.binopinfo GrB.binopinfo (op) GrB.binopinfo (op, optype) Binary operators are defined by a string of the form 'op.optype', or just 'op', where the optype is inferred from the operands. Valid optypes are 'logical', 'int8', 'int16', 'int32', 'int64', 'uint8', 'uint16', 'uint32', 'uint64', 'single', 'double', 'single complex', 'double complex' (the latter can be written as simply 'complex'). For GrB.binopinfo (op), the op must be a string of the form 'op.optype', where 'op' is listed below. The second usage allows the optype to be omitted from the first argument, as just 'op'. This is valid for all GraphBLAS operations, since the optype can be determined from the operands (see Typecasting, below). However, GrB.binopinfo does not have any operands and thus the optype must be provided, either in the op as GrB.binopinfo ('+.double'), or in the second argument as GrB.binopinfo ('+', 'double'). The 6 comparator operators come in two flavors. For the is* operators, the result has the same type as the inputs, x and y, with 1 for true and 0 for false. For example isgt.double (pi, 3.0) is the double value 1.0. For the second set of 6 operators (eq, ne, gt, lt, ge, le), the result is always logical (true or false). In a semiring, the optype of the add monoid must exactly match the type of the output of the multiply operator, and thus 'plus.iseq.double' is valid (counting how many terms are equal). The 'plus.eq.double' semiring is valid, but not the same semiring since the 'plus' of 'plus.eq.double' has a logical type and is thus equivalent to 'or.eq.double'. The 'or.eq' is true if any terms are equal and false otherwise (it does not count the number of terms that are equal). The following binary operators are available for most types. Many have equivalent synonyms, so that '1st' and 'first' both define the first(x,y) = x operator. operator name(s) f(x,y) | operator names(s) f(x,y) ---------------- ------ | ----------------- ------ 1st first x | iseq x == y 2nd second y | isne x ~= y min min(x,y) | isgt x > y max max(x,y) | islt x < y + plus x+y | isge x >= y - minus x-y | isle x <= y rminus y-x | == eq x == y * times x*y | ~= ne x ~= y / div x/y | > gt x > y \ rdiv y/x | < lt x < y | || or lor x | y | >= ge x >= y & && and land x & y | <= le x <= y xor lxor xor(x,y) | .^ pow x .^ y pair 1 | any pick x or y Comparators (*lt, *gt, *le, *ge) and min/max are not available for complex types. All of the above operators are defined for logical operands, but many are redundant. 'min.logical' is the same as 'and.logical', for example. Most of the logical operators have aliases: ('lor', 'or', '|') are the same, as are ('lxnor', 'xnor', 'eq', '==') for logical types. The three logical operators, lor, land, and lxor, can be used with any real types. z = lor.double (x,y) tests the condition (x~=0) || (y~=0), and returns the double value 1.0 if true, or 0.0 if false. The following operators are avaiable for single and double (real); their definitions are identical to the ANSI C11 versions of these functions: atan2, hypot, fmod, remainder, copysign, ldxep (also called 'pow2'). All produce the same type as the input, on output. z = cmplx(x,y) can be computed for x and y as single and double; z is single complex or double complex, respectively. The following bitwise operators are available for any signed or unsigned integer types: bitor, bitand, bitxor, bitxnor, bitget, bitset, bitclr, and bitshift. Typecasting: If the optype is omitted from the string (for example, GrB.eadd (A, '+', B) or simply C = A+B), then the optype is inferred from the type of A and B. See 'help GrB.optype' for details. Example: % valid binary operators GrB.binopinfo ('+.double') ; % also a valid unary operator GrB.binopinfo ('1st.int32') ; GrB.binopinfo ('cmplx.single') ; GrB.binopinfo ('pow2.double') ; % also a valid unary operator GrB.unopinfo ('pow2.double') ; % invalid binary operator (an error; this is a unary op): GrB.binopinfo ('abs.double') ; See also GrB.descriptorinfo, GrB.monoidinfo, GrB.selectopinfo, GrB.semiringinfo, GrB.unopinfo, GrB.optype.
help GrB.monoidinfo
GRB.MONOIDINFO list the details of a GraphBLAS monoid. GrB.monoidinfo GrB.monoidinfo (monoid) GrB.monoidinfo (monoid, type) For GrB.monoidinfo(op), the op must be a string of the form 'op.type', where 'op' is listed below. The second usage allows the type to be omitted from the first argument, as just 'op'. This is valid for all GraphBLAS operations, since the type defaults to the type of the input matrices. However, GrB.monoidinfo does not have a default type and thus one must be provided, either in the op as GrB.monoidinfo ('+.double'), or in the second argument, GrB.monoidinfo ('+', 'double'). A monoid is any binary operator z=f(x,y) that is commutative and associative, with an identity value o so that f(x,o)=f(o,x)=o. The types of z, x, and y must all be identical. For example, the plus.double operator is f(x,y)=x+y, with zero as the identity value (x+0 = 0+x = x). The times monoid has an identity value of 1 (since x*1 = 1*x = x). The identity of min.double is -inf. The valid monoids for real non-logical types are: '+', '*', 'max', 'min', 'any' For the 'logical' type: '|', '&', 'xor', 'eq', 'any' For complex types: '+', '*', 'any' For integer types (signed and unsigned): 'bitor', 'bitand', 'bitxor', 'bitxnor' Some monoids have synonyms; see 'help GrB.binopinfo' for details. Example: % valid monoids GrB.monoidinfo ('+.double') ; GrB.monoidinfo ('*.int32') ; GrB.monoidinfo ('min.double') ; % invalid monoids GrB.monoidinfo ('1st.int32') ; GrB.monoidinfo ('abs.double') ; GrB.monoidinfo ('min.complex') ; See also GrB.binopinfo, GrB.descriptorinfo, GrB.selectopinfo, GrB.semiringinfo, GrB.unopinfo.
help GrB.unopinfo
GRB.UNOPINFO list the details of a GraphBLAS unary operator. GrB.unopinfo GrB.unopinfo (op) GrB.unopinfo (op, type) For GrB.unopinfo(op), the op must be a string of the form 'op.type', where 'op' is listed below. The second usage allows the type to be omitted from the first argument, as just 'op'. This is valid for all GraphBLAS operations, since the type defaults to the type of the input matrix. However, GrB.unopinfo does not have a default type and thus one must be provided, either in the op as GrB.unopinfo ('abs.double'), or in the second argument, GrB.unopinfo ('abs', 'double'). The functions z=f(x) are listed below. Unless otherwise specified, z and x have the same type. Some functions have synonyms, as listed. For all 13 types: identity z = x also '+', 'uplus' ainv z = -x additive inverse, also '-', 'negate', 'uminus' minv z = 1/x multiplicative inverse one z = 1 does not depend on x, also '1' abs z = |x| 'abs.complex' returns a real result For all 11 real types: lnot z = ~(x ~= 0) logical negation (z is 1 or 0, with the same type as x), also '~', 'not'. For 4 floating-point types (real & complex)x(single & double): sqrt z = sqrt (x) square root log z = log (x) base-e logarithm log2 z = log2 (x) base-2 logarithm log10 z = log10 (x) base-10 logarithm log1p z = log1p (x) log (x-1), base-e exp z = exp (x) base-e exponential, e^x pow2 z = pow2 (x) base-2 exponential, 2^x expm1 z = exp1m (x) e^x-1 sin z = sin (x) sine cos z = cos (x) cosine tan z = tan (x) tangent acos z = acos (x) arc cosine asin z = asin (x) arc sine atan z = atan (x) arc tangent sinh z = sinh (x) hyperbolic sine cosh z = cosh (x) hyperbolic cosine tanh z = tanh (x) hyperbolic tangent asinh z = asinh (x) inverse hyperbolic sine acosh z = acosh (x) inverse hyperbolic cosine atanh z = atanh (x) inverse hyperbolic tangent signum z = signum (x) signum function, also 'sign' ceil z = ceil (x) ceiling floor z = floor (x) floor round z = round (x) round to nearest trunc z = trunc (x) truncate, also 'fix' For 'single complex' and 'double complex' only: creal z = real (x) real part of x (z is real), also 'real' cimag z = imag (x) imag. part of x (z is real), also 'imag' carg z = carg (x) phase angle (z is real), also 'angle' conj z = conj (x) complex conjugate (z is complex) For all 4 floating-point types (result is logical): isinf z = isinf (x) true if x is +Inf or -Inf isnan z = isnan (x) true if x is NaN isfinite z = isfinite (x) true if x is finite For single and double (result same type as input): lgamma z = lgamma (x) log of gamma function, also 'gammaln' tgamma z = tgamma (x) gamma function, also 'gamma' erf z = erf (x) error function erfc z = erfc (x) complementary error function frexpx z = frexpx (x) mantissa from ANSI C11 frexp function frexpe z = frexpe (x) exponent from ANSI C11 frexp function The MATLAB [f,e]=log2(x) returns f = frexpx (x) and e = frexpe (x). For integer types only (result is same type as input): bitcmp z = ~(x) bitwise complement, also 'bitnot' Example: % valid unary operators GrB.unopinfo ('+.double') ; % also a valid binary operator GrB.unopinfo ('abs.double') ; GrB.unopinfo ('not.int32') ; GrB.unopinfo ('pow2.double') ; % also a valid binary operator GrB.binopinfo ('pow2.double') ; % invalid unary operator (generates an error; this is a binary op): GrB.unopinfo ('*.double') ; See also GrB.binopinfo, GrB.descriptorinfo, GrB.monoidinfo, GrB.selectopinfo, GrB.semiringinfo.
Element-wise operations
Binary operators can be used in element-wise matrix operations, like C=A+B and C=A.*B. For the matrix addition C=A+B, the pattern of C is the set union of A and B, and the '+' operator is applied for entries in the intersection. Entries in A but not B, or in B but not A, are assigned to C without using the operator. The '+' operator is used for C=A+B but any operator can be used with GrB.eadd.
A = GrB (sprand (3, 3, 0.5)) ;
B = GrB (sprand (3, 3, 0.5)) ;
C1 = A + B
C2 = GrB.eadd ('+', A, B)
err = norm (C1-C2,1)
C1 = 3x3 GraphBLAS double matrix, sparse by col: 7 nonzeros, 7 entries (1,1) 0.666139 (3,1) 0.735859 (1,2) 1.47841 (2,2) 0.146938 (3,2) 0.566879 (2,3) 0.248635 (3,3) 0.104226 C2 = 3x3 GraphBLAS double matrix, sparse by col: 7 nonzeros, 7 entries (1,1) 0.666139 (3,1) 0.735859 (1,2) 1.47841 (2,2) 0.146938 (3,2) 0.566879 (2,3) 0.248635 (3,3) 0.104226 err = 0
Subtracting two matrices
A-B and GrB.eadd ('-', A, B) are not the same thing, since the '-' operator is not applied to an entry that is in B but not A.
C1 = A-B
C2 = GrB.eadd ('-', A, B)
C1 = 3x3 GraphBLAS double matrix, sparse by col: 7 nonzeros, 7 entries (1,1) -0.666139 (3,1) -0.735859 (1,2) -0.334348 (2,2) -0.146938 (3,2) 0.566879 (2,3) 0.248635 (3,3) 0.104226 C2 = 3x3 GraphBLAS double matrix, sparse by col: 7 nonzeros, 7 entries (1,1) 0.666139 (3,1) 0.735859 (1,2) -0.334348 (2,2) 0.146938 (3,2) 0.566879 (2,3) 0.248635 (3,3) 0.104226
But these give the same result
C1 = A-B C2 = GrB.eadd ('+', A, GrB.apply ('-', B)) err = norm (C1-C2,1)
C1 = 3x3 GraphBLAS double matrix, sparse by col: 7 nonzeros, 7 entries (1,1) -0.666139 (3,1) -0.735859 (1,2) -0.334348 (2,2) -0.146938 (3,2) 0.566879 (2,3) 0.248635 (3,3) 0.104226 C2 = 3x3 GraphBLAS double matrix, sparse by col: 7 nonzeros, 7 entries (1,1) -0.666139 (3,1) -0.735859 (1,2) -0.334348 (2,2) -0.146938 (3,2) 0.566879 (2,3) 0.248635 (3,3) 0.104226 err = 0
Element-wise 'multiplication'
For C = A.*B, the result C is the set intersection of the pattern of A and B. The operator is applied to entries in both A and B. Entries in A but not B, or B but not A, do not appear in the result C.
C1 = A.*B
C2 = GrB.emult ('*', A, B)
C3 = double (A) .* double (B)
C1 = 3x3 GraphBLAS double matrix, sparse by col: 1 nonzero, 1 entry (1,2) 0.518474 C2 = 3x3 GraphBLAS double matrix, sparse by col: 1 nonzero, 1 entry (1,2) 0.518474 C3 = (1,2) 0.518474419030681
Just as in GrB.eadd, any operator can be used in GrB.emult:
A
B
C2 = GrB.emult ('max', A, B)
A = 3x3 GraphBLAS double matrix, sparse by col: 4 nonzeros, 4 entries (1,2) 0.572029 (3,2) 0.566879 (2,3) 0.248635 (3,3) 0.104226 B = 3x3 GraphBLAS double matrix, sparse by col: 4 nonzeros, 4 entries (1,1) 0.666139 (3,1) 0.735859 (1,2) 0.906378 (2,2) 0.146938 C2 = 3x3 GraphBLAS double matrix, sparse by col: 1 nonzero, 1 entry (1,2) 0.906378
Overloaded operators
The following operators all work as you would expect for any matrix. The matrices A and B can be GraphBLAS matrices, or MATLAB sparse or dense matrices, in any combination, or scalars where appropriate, The matrix M is logical (MATLAB or GraphBLAS):
A+B A-B A*B A.*B A./B A.\B A.^b A/b C=A(I,J) C(M)=A -A +A ~A A' A.' A&B A|B b\A C(I,J)=A C=A(M) A~=B A>B A==B A<=B A>=B A<B [A,B] [A;B] C(A) A(1:end,1:end)
For A^b, b must be a non-negative integer.
C1 = [A B] ; C2 = [double(A) double(B)] ; assert (isequal (double (C1), C2))
C1 = A^2 C2 = double (A)^2 ; err = norm (C1 - C2, 1) assert (err < 1e-12)
C1 = 3x3 GraphBLAS double matrix, sparse by col: 5 nonzeros, 5 entries (2,2) 0.140946 (3,2) 0.0590838 (1,3) 0.142227 (2,3) 0.0259144 (3,3) 0.151809 err = 0
C1 = A (1:2,2:end) A = double (A) ; C2 = A (1:2,2:end) ; assert (isequal (double (C1), C2))
C1 = 2x2 GraphBLAS double matrix, sparse by col: 2 nonzeros, 2 entries (1,1) 0.572029 (2,2) 0.248635
Overloaded functions
Many MATLAB built-in functions can be used with GraphBLAS matrices:
A few differences with the built-in functions:
S = sparse (G) % makes a copy of a GrB matrix F = full (G) % adds explicit zeros, so numel(F)==nnz(F) F = full (G,type,id) % adds explicit identity values to a GrB matrix disp (G, level) % display a GrB matrix G; level=2 is the default.
In the list below, the first set of Methods are overloaded built-in methods. They are used as-is on GraphBLAS matrices, such as C=abs(G). The Static methods are prefixed with "GrB.", as in C = GrB.apply ( ... ).
methods GrB
Methods for class GrB: GrB disp islogical real abs display ismatrix repmat acos dmperm isnan reshape acosh double isnumeric round acot eig isreal sec acoth end isscalar sech acsc eps issparse sign acsch eq issymmetric sin all erf istril single amd erfc istriu sinh and etree isvector size angle exp kron sparse any expm1 ldivide spfun asec false le spones asech find length sprand asin fix log sprandn asinh flip log10 sprandsym assert floor log1p sprintf atan fprintf log2 sqrt atan2 full logical subsasgn atanh gamma lt subsindex bandwidth gammaln max subsref bitand ge min sum bitcmp graph minus symamd bitget gt mldivide symrcm bitor horzcat mpower tan bitset hypot mrdivide tanh bitshift imag mtimes times bitxor int16 ne transpose ceil int32 nnz tril colamd int64 nonzeros triu complex int8 norm true conj isa not uint16 cos isbanded numel uint32 cosh isdiag nzmax uint64 cot isempty ones uint8 coth isequal or uminus csc isfinite plus uplus csch isfloat pow2 vertcat ctranspose ishermitian power xor diag isinf prod zeros digraph isinteger rdivide Static methods: MATLAB_vs_GrB empty issigned reduce apply emult kronecker select apply2 entries ktruss selectopinfo assign expand laplacian semiringinfo bfs extract mis speye binopinfo extracttuples monoidinfo subassign build eye mxm threads burble finalize nonz trans chunk format normdiff tricount clear incidence offdiag type compact init optype unopinfo descriptorinfo isbycol pagerank vreduce dnn isbyrow prune eadd isfull random
Zeros are handled differently
Explicit zeros cannot be automatically dropped from a GraphBLAS matrix, like they are in MATLAB sparse matrices. In a shortest-path problem, for example, an edge A(i,j) that is missing has an infinite weight, (the monoid identity of min(x,y) is +inf). A zero edge weight A(i,j)=0 is very different from an entry that is not present in A. However, if a GraphBLAS matrix is converted into a MATLAB sparse matrix, explicit zeros are dropped, which is the convention for a MATLAB sparse matrix. They can also be dropped from a GraphBLAS matrix using the GrB.select method.
G = GrB (magic (2)) ; G (1,1) = 0 % G(1,1) still appears as an explicit entry A = double (G) % but it's dropped when converted to MATLAB sparse H = GrB.select ('nonzero', G) % drops the explicit zeros from G fprintf ('nnz (G): %d nnz (A): %g nnz (H): %g\n', ... nnz (G), nnz (A), nnz (H)) ; fprintf ('num entries in G: %d\n', GrB.entries (G)) ;
G = 2x2 GraphBLAS double matrix, sparse by col: 3 nonzeros, 4 entries (1,1) 0 (2,1) 4 (1,2) 3 (2,2) 2 A = (2,1) 4 (1,2) 3 (2,2) 2 H = 2x2 GraphBLAS double matrix, sparse by col: 3 nonzeros, 3 entries (2,1) 4 (1,2) 3 (2,2) 2 nnz (G): 3 nnz (A): 3 nnz (H): 3 num entries in G: 4
Displaying contents of a GraphBLAS matrix
Unlike MATLAB, the default is to display just a few entries of a GrB matrix. Here are all 100 entries of a 10-by-10 matrix, using a non-default disp(G,3):
G = GrB (rand (10)) ;
% display everything:
disp (G,3)
G = 10x10 GraphBLAS double matrix, sparse by col: 100 nonzeros, 100 entries (1,1) 0.0342763 (2,1) 0.17802 (3,1) 0.887592 (4,1) 0.889828 (5,1) 0.769149 (6,1) 0.00497062 (7,1) 0.735693 (8,1) 0.488349 (9,1) 0.332817 (10,1) 0.0273313 (1,2) 0.467212 (2,2) 0.796714 (3,2) 0.849463 (4,2) 0.965361 (5,2) 0.902248 (6,2) 0.0363252 (7,2) 0.708068 (8,2) 0.322919 (9,2) 0.700716 (10,2) 0.472957 (1,3) 0.204363 (2,3) 0.00931977 (3,3) 0.565881 (4,3) 0.183435 (5,3) 0.00843818 (6,3) 0.284938 (7,3) 0.706156 (8,3) 0.909475 (9,3) 0.84868 (10,3) 0.564605 (1,4) 0.075183 (2,4) 0.535293 (3,4) 0.072324 (4,4) 0.515373 (5,4) 0.926149 (6,4) 0.949252 (7,4) 0.0478888 (8,4) 0.523767 (9,4) 0.167203 (10,4) 0.28341 (1,5) 0.122669 (2,5) 0.441267 (3,5) 0.157113 (4,5) 0.302479 (5,5) 0.758486 (6,5) 0.910563 (7,5) 0.0246916 (8,5) 0.232421 (9,5) 0.38018 (10,5) 0.677531 (1,6) 0.869074 (2,6) 0.471459 (3,6) 0.624929 (4,6) 0.987186 (5,6) 0.282885 (6,6) 0.843833 (7,6) 0.869597 (8,6) 0.308209 (9,6) 0.201332 (10,6) 0.706603 (1,7) 0.563222 (2,7) 0.575795 (3,7) 0.056376 (4,7) 0.73412 (5,7) 0.608022 (6,7) 0.0400164 (7,7) 0.540801 (8,7) 0.023064 (9,7) 0.165682 (10,7) 0.250393 (1,8) 0.23865 (2,8) 0.232033 (3,8) 0.303191 (4,8) 0.579934 (5,8) 0.267751 (6,8) 0.916376 (7,8) 0.833499 (8,8) 0.978692 (9,8) 0.734445 (10,8) 0.102896 (1,9) 0.353059 (2,9) 0.738955 (3,9) 0.57539 (4,9) 0.751433 (5,9) 0.93256 (6,9) 0.281622 (7,9) 0.51302 (8,9) 0.24406 (9,9) 0.950086 (10,9) 0.303638 (1,10) 0.563593 (2,10) 0.705101 (3,10) 0.0604146 (4,10) 0.672065 (5,10) 0.359793 (6,10) 0.62931 (7,10) 0.977758 (8,10) 0.394328 (9,10) 0.765651 (10,10) 0.457809
That was disp(G,3), so every entry was printed. It's a little long, so the default is not to print everything.
With the default display (level = 2):
G
G = 10x10 GraphBLAS double matrix, sparse by col: 100 nonzeros, 100 entries (1,1) 0.0342763 (2,1) 0.17802 (3,1) 0.887592 (4,1) 0.889828 (5,1) 0.769149 (6,1) 0.00497062 (7,1) 0.735693 (8,1) 0.488349 (9,1) 0.332817 (10,1) 0.0273313 (1,2) 0.467212 (2,2) 0.796714 (3,2) 0.849463 (4,2) 0.965361 (5,2) 0.902248 (6,2) 0.0363252 (7,2) 0.708068 (8,2) 0.322919 (9,2) 0.700716 (10,2) 0.472957 (1,3) 0.204363 (2,3) 0.00931977 (3,3) 0.565881 (4,3) 0.183435 (5,3) 0.00843818 (6,3) 0.284938 (7,3) 0.706156 (8,3) 0.909475 (9,3) 0.84868 (10,3) 0.564605 ...
That was disp(G,2) or just display(G), which is what is printed by a MATLAB statement that doesn't have a trailing semicolon. With level = 1, disp(G,1) gives just a terse summary:
disp (G,1)
G = 10x10 GraphBLAS double matrix, sparse by col: 100 nonzeros, 100 entries
Storing a matrix by row or by column
MATLAB stores its sparse matrices by column, refered to as 'standard CSC' in SuiteSparse:GraphBLAS. In the CSC (compressed sparse column) format, each column of the matrix is stored as a list of entries, with their value and row index. In the CSR (compressed sparse row) format, each row is stored as a list of values and their column indices. GraphBLAS uses both CSC and CSR, and the two formats can be intermixed arbitrarily. In its C interface, the default format is CSR. However, for better compatibility with MATLAB, this MATLAB interface for SuiteSparse:GraphBLAS uses CSC by default instead.
rng ('default') ; GrB.clear ; % clear prior GraphBLAS settings fprintf ('the default format is: %s\n', GrB.format) ; C = sparse (rand (2)) G = GrB (C) GrB.format (G)
the default format is: by col C = (1,1) 0.814723686393179 (2,1) 0.905791937075619 (1,2) 0.126986816293506 (2,2) 0.913375856139019 G = 2x2 GraphBLAS double matrix, sparse by col: 4 nonzeros, 4 entries (1,1) 0.814724 (2,1) 0.905792 (1,2) 0.126987 (2,2) 0.913376 ans = 'by col'
Many graph algorithms work better in CSR format, with matrices stored by row. For example, it is common to use A(i,j) for the edge (i,j), and many graph algorithms need to access the out-adjacencies of nodes, which is the row A(i,;) for node i. If the CSR format is desired, GrB.format ('by row') tells GraphBLAS to create all subsequent matrices in the CSR format. Converting from a MATLAB sparse matrix (in standard CSC format) takes a little more time (requiring a transpose), but subsequent graph algorithms can be faster.
G = GrB (C, 'by row') fprintf ('the format of G is: %s\n', GrB.format (G)) ; H = GrB (C) fprintf ('the format of H is: %s\n', GrB.format (H)) ; err = norm (H-G,1)
G = 2x2 GraphBLAS double matrix, sparse by row: 4 nonzeros, 4 entries (1,1) 0.814724 (1,2) 0.126987 (2,1) 0.905792 (2,2) 0.913376 the format of G is: by row H = 2x2 GraphBLAS double matrix, sparse by col: 4 nonzeros, 4 entries (1,1) 0.814724 (2,1) 0.905792 (1,2) 0.126987 (2,2) 0.913376 the format of H is: by col err = 0
Hypersparse matrices
SuiteSparse:GraphBLAS can use two kinds of sparse matrix data structures: standard and hypersparse, for both CSC and CSR formats. In the standard CSC format used in MATLAB, an m-by-n matrix A takes O(n+nnz(A)) space. MATLAB can create huge column vectors, but not huge matrices (when n is huge).
clear [c, huge] = computer ; C = sparse (huge, 1) % MATLAB can create a huge-by-1 sparse column try C = sparse (huge, huge) % but this fails catch me error_expected = me end
C = All zero sparse: 281474976710655×1 error_expected = MException with properties: identifier: 'MATLAB:array:SizeLimitExceeded' message: 'Requested 281474976710655x281474976710655 (2097152.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See <a href="matlab: helpview([docroot '/matlab/helptargets.map'], 'matlab_env_workspace_prefs')">array size limit</a> or preference panel for more information.' cause: {} stack: [4×1 struct] Correction: []
In a GraphBLAS hypersparse matrix, an m-by-n matrix A takes only O(nnz(A)) space. The difference can be huge if nnz (A) << n.
clear [c, huge] = computer ; G = GrB (huge, 1) % no problem for GraphBLAS H = GrB (huge, huge) % this works in GraphBLAS too
G = 281474976710655x1 GraphBLAS double matrix, sparse by col: no nonzeros, no entries H = 281474976710655x281474976710655 GraphBLAS double matrix, hypersparse by col: no nonzeros, no entries
Operations on huge hypersparse matrices are very fast; no component of the time or space complexity is Omega(n).
I = randperm (huge, 2) ; J = randperm (huge, 2) ; H (I,J) = magic (2) ; % add 4 nonzeros to random locations in H H (I,I) = 10 * [1 2 ; 3 4] ; % so H^2 is not all zero H = H^2 ; % square H H = (H' * 2) ; % transpose H and double the entries K = pi * spones (H) ; H = H + K % add pi to each entry in H
H = 281474976710655x281474976710655 GraphBLAS double matrix, hypersparse by col: 8 nonzeros, 8 entries (27455183225557,27455183225557) 4403.14 (78390279669562,27455183225557) 383.142 (153933462881710,27455183225557) 343.142 (177993304104065,27455183225557) 3003.14 (27455183225557,177993304104065) 2003.14 (78390279669562,177993304104065) 183.142 (153933462881710,177993304104065) 143.142 (177993304104065,177993304104065) 1403.14
numel uses vpa if the matrix is really huge
e1 = numel (G) % this is huge, but still a flint e2 = numel (H) % this is huge^2, which needs vpa whos e1 e2
e1 = 281474976710655 e2 = 79228162514263774643590529025.0 Name Size Bytes Class Attributes e1 1x1 8 double e2 1x1 8 sym
All of these matrices take very little memory space:
whos C G H K
Name Size Bytes Class Attributes G 281474976710655x1 981 GrB H 281474976710655x281474976710655 1300 GrB K 281474976710655x281474976710655 1300 GrB
The mask and accumulator
When not used in overloaded operators or built-in functions, many GraphBLAS methods of the form GrB.method ( ... ) can optionally use a mask and/or an accumulator operator. If the accumulator is '+' in GrB.mxm, for example, then C = C + A*B is computed. The mask acts much like logical indexing in MATLAB. With a logical mask matrix M, C<M>=A*B allows only part of C to be assigned. If M(i,j) is true, then C(i,j) can be modified. If false, then C(i,j) is not modified.
For example, to set all values in C that are greater than 0.5 to 3:
A = rand (3) C = GrB.assign (A, A > 0.5, 3) ; % in GraphBLAS C1 = GrB (A) ; C1 (A > .5) = 3 % also in GraphBLAS C2 = A ; C2 (A > .5) = 3 % in MATLAB err = norm (C - C1, 1) err = norm (C - C2, 1)
A = 0.957506835434298 0.970592781760616 0.8002804688888 0.964888535199277 0.957166948242946 0.141886338627215 0.157613081677548 0.485375648722841 0.421761282626275 C1 = 3x3 GraphBLAS double matrix, sparse by col: 9 nonzeros, 9 entries (1,1) 3 (2,1) 3 (3,1) 0.157613 (1,2) 3 (2,2) 3 (3,2) 0.485376 (1,3) 3 (2,3) 0.141886 (3,3) 0.421761 C2 = 3 3 3 3 3 0.141886338627215 0.157613081677548 0.485375648722841 0.421761282626275 err = 0 err = 0
The descriptor
Most GraphBLAS functions of the form GrB.method ( ... ) take an optional last argument, called the descriptor. It is a MATLAB struct that can modify the computations performed by the method. 'help GrB.descriptorinfo' gives all the details. The following is a short summary of the primary settings:
d.out = 'default' or 'replace', clears C after the accum op is used.
d.mask = 'default' or 'complement', to use M or ~M as the mask matrix; 'structural', or 'structural complement', to use the pattern of M or ~M.
d.in0 = 'default' or 'transpose', to transpose A for C=A*B, C=A+B, etc.
d.in1 = 'default' or 'transpose', to transpose B for C=A*B, C=A+B, etc.
d.kind = 'default', 'GrB', 'sparse', or 'full'; the output of GrB.method.
A = sparse (rand (2)) ; B = sparse (rand (2)) ; C1 = A'*B ; C2 = GrB.mxm ('+.*', A, B, struct ('in0', 'transpose')) ; err = norm (C1-C2,1)
err = 0
Integer arithmetic is different in GraphBLAS
MATLAB supports integer arithmetic on its full matrices, using int8, int16, int32, int64, uint8, uint16, uint32, or uint64 data types. None of these integer data types can be used to construct a MATLAB sparse matrix, which can only be double, double complex, or logical. Furthermore, C=A*B is not defined for integer types in MATLAB, except when A and/or B are scalars.
GraphBLAS supports all of those types for its sparse matrices. All operations are supported, including C=A*B when A or B are any integer type, in 1000s of semirings.
However, integer arithmetic differs in GraphBLAS and MATLAB. In MATLAB, integer values saturate if they exceed their maximum value. In GraphBLAS, integer operators act in a modular fashion. The latter is essential when computing C=A*B over a semiring. A saturating integer operator cannot be used as a monoid since it is not associative.
C = uint8 (magic (3)) ; G = GrB (C) ; C1 = C * 40 C2 = G * uint8 (40) S = double (C1 < 255) ; assert (isequal (double (C1).*S, double (C2).*S))
C1 = 3×3 uint8 matrix 255 40 240 120 200 255 160 255 80 C2 = 3x3 GraphBLAS uint8_t matrix, sparse by col: 9 nonzeros, 9 entries (1,1) 64 (2,1) 120 (3,1) 160 (1,2) 40 (2,2) 200 (3,2) 104 (1,3) 240 (2,3) 24 (3,3) 80
An example graph algorithm: breadth-first search
The breadth-first search of a graph finds all nodes reachable from the source node, and their level, v. v=GrB.bfs(A,s) or v=bfs_matlab(A,s) compute the same thing, but GrB.bfs uses GraphBLAS matrices and operations, while bfs_matlab uses pure MATLAB operations. v is defined as v(s) = 1 for the source node, v(i) = 2 for nodes adjacent to the source, and so on.
clear rng ('default') ; n = 1e5 ; A = logical (sprandn (n, n, 1e-3)) ; tic v1 = GrB.bfs (A, 1) ; gb_time = toc ; tic v2 = bfs_matlab (A, 1) ; matlab_time = toc ; assert (isequal (double (v1'), v2)) fprintf ('\nnodes reached: %d of %d\n', nnz (v2), n) ; fprintf ('GraphBLAS time: %g sec\n', gb_time) ; fprintf ('MATLAB time: %g sec\n', matlab_time) ; fprintf ('Speedup of GraphBLAS over MATLAB: %g\n', ... matlab_time / gb_time) ;
nodes reached: 100000 of 100000 GraphBLAS time: 0.33517 sec MATLAB time: 0.799412 sec Speedup of GraphBLAS over MATLAB: 2.3851
Example graph algorithm: Luby's method in GraphBLAS
The GrB.mis function is variant of Luby's randomized algorithm [Luby 1985]. It is a parallel method for finding an maximal independent set of nodes, where no two nodes are adjacent. See the GraphBLAS/@GrB/mis.m function for details. The graph must be symmetric with a zero-free diagonal, so A is symmetrized first and any diagonal entries are removed.
A = GrB (A) ; A = GrB.offdiag (A|A') ; tic s = GrB.mis (A) ; toc fprintf ('# nodes in the graph: %g\n', size (A,1)) ; fprintf ('# edges: : %g\n', GrB.entries (A) / 2) ; fprintf ('size of maximal independent set found: %g\n', ... full (double (sum (s)))) ; % make sure it's independent p = find (s) ; S = A (p,p) ; assert (GrB.entries (S) == 0) % make sure it's maximal notp = find (s == 0) ; S = A (notp, p) ; deg = GrB.vreduce ('+.int64', S) ; assert (logical (all (deg > 0)))
Elapsed time is 0.542797 seconds. # nodes in the graph: 100000 # edges: : 9.9899e+06 size of maximal independent set found: 2811
Sparse deep neural network
The 2019 MIT GraphChallenge (see http://graphchallenge.org) is to solve a set of large sparse deep neural network problems. In this demo, the MATLAB reference solution is compared with a solution using GraphBLAS, for a randomly constructed neural network. See the GrB.dnn and dnn_matlab.m functions for details.
clear rng ('default') ; nlayers = 16 ; nneurons = 4096 ; nfeatures = 30000 ; fprintf ('# layers: %d\n', nlayers) ; fprintf ('# neurons: %d\n', nneurons) ; fprintf ('# features: %d\n', nfeatures) ; tic Y0 = sprand (nfeatures, nneurons, 0.1) ; for layer = 1:nlayers W {layer} = sprand (nneurons, nneurons, 0.01) * 0.2 ; bias {layer} = -0.2 * ones (1, nneurons) ; end t_setup = toc ; fprintf ('construct problem time: %g sec\n', t_setup) ; % convert the problem from MATLAB to GraphBLAS t = tic ; [W_gb, bias_gb, Y0_gb] = dnn_mat2gb (W, bias, Y0) ; t = toc (t) ; fprintf ('setup time: %g sec\n', t) ;
# layers: 16 # neurons: 4096 # features: 30000 construct problem time: 6.02037 sec setup time: 0.284302 sec
Solving the sparse deep neural network problem with GraphbLAS
Please wait ...
tic
Y1 = GrB.dnn (W_gb, bias_gb, Y0_gb) ;
gb_time = toc ;
fprintf ('total time in GraphBLAS: %g sec\n', gb_time) ;
total time in GraphBLAS: 12.7092 sec
Solving the sparse deep neural network problem with MATLAB
Please wait ...
tic Y2 = dnn_matlab (W, bias, Y0) ; matlab_time = toc ; fprintf ('total time in MATLAB: %g sec\n', matlab_time) ; fprintf ('Speedup of GraphBLAS over MATLAB: %g\n', ... matlab_time / gb_time) ; err = norm (Y1-Y2,1)
total time in MATLAB: 105.194 sec Speedup of GraphBLAS over MATLAB: 8.27702 err = 0
For objects, GraphBLAS has better colon notation than MATLAB
The MATLAB notation C = A (start:inc:fini) is very handy, and it works great if A is a MATLAB matrix. But for objects like the GraphBLAS matrix, MATLAB starts by creating the explicit index vector I = start:inc:fini. That's fine if the matrix is modest in size, but GraphBLAS can construct huge matrices. The problem is that 1:n cannot be explicitly constructed when n is huge.
The C API for GraphBLAS can represent the colon notation start:inc:fini in an implicit manner, so it can do the indexing without actually forming the explicit list I = start:inc:fini. But there is no access to this method using the MATLAB notation start:inc:fini.
Thus, to compute C = A (start:inc:fini) for very huge matrices, you need to use use a cell array to represent the colon notation, as { start, inc, fini }, instead of start:inc:fini. See 'help GrB.extract', 'help GrB.assign' for the functional form. For the overloaded syntax C(I,J)=A and C=A(I,J), see 'help GrB/subsasgn' and 'help GrB/subsfref'. The cell array syntax isn't conventional, but it is far faster than the MATLAB colon notation for objects, and takes far less memory when I is huge.
n = 1e14 ; H = GrB (n, n) ; % a huge empty matrix I = [1 1e9 1e12 1e14] ; M = magic (4) H (I,I) = M ; J = {1, 1e13} ; % represents 1:1e13 colon notation C1 = H (J, J) % computes C1 = H (1:e13,1:1e13) c = nonzeros (C1) ; m = nonzeros (M (1:3, 1:3)) ; assert (isequal (c, m)) ;
M = 16 2 3 13 5 11 10 8 9 7 6 12 4 14 15 1 C1 = 10000000000000x10000000000000 GraphBLAS double matrix, hypersparse by col: 9 nonzeros, 9 entries (1,1) 16 (1000000000,1) 5 (1000000000000,1) 9 (1,1000000000) 2 (1000000000,1000000000) 11 (1000000000000,1000000000) 7 (1,1000000000000) 3 (1000000000,1000000000000) 10 (1000000000000,1000000000000) 6
try % try to compute the same thing with colon % notation (1:1e13), but this fails: C2 = H (1:1e13, 1:1e13) catch me error_expected = me end
error_expected = MException with properties: identifier: 'MATLAB:array:SizeLimitExceeded' message: 'Requested 10000000000000x1 (74505.8GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See <a href="matlab: helpview([docroot '/matlab/helptargets.map'], 'matlab_env_workspace_prefs')">array size limit</a> or preference panel for more information.' cause: {} stack: [4×1 struct] Correction: []
Iterative solvers work as-is
Many built-in functions work with GraphBLAS matrices unmodified.
A = sparse (rand (4)) ; b = sparse (rand (4,1)) ; x = gmres (A,b) norm (A*x-b) x = gmres (GrB(A), GrB(b)) norm (A*x-b)
gmres converged at iteration 4 to a solution with relative residual 0. x = 0.91047557490445 3.89492850585206 -0.569511178041979 -1.38669683854936 ans = 8.67111901826273e-16 gmres converged at iteration 4 to a solution with relative residual 0. x = 0.910475574904404 3.8949285058519 -0.56951117804191 -1.3866968385493 ans = 7.28021923224409e-16
... even in single precision
x = gmres (GrB(A,'single'), GrB(b,'single')) norm (A*x-b)
gmres converged at iteration 4 to a solution with relative residual 0. x = 0.910472507135207 3.89491683958708 -0.569506481916661 -1.3866920717142 ans = 8.3369210904823e-08
Both of the following uses of minres (A,b) fail to converge because A is not symmetric, as the method requires. Both failures are correctly reported, and both the MATLAB version and the GraphBLAS version return the same incorrect vector x.
x = minres (A, b) x = minres (GrB(A), GrB(b))
minres stopped at iteration 4 without converging to the desired tolerance 1e-06 because the maximum number of iterations was reached. The iterate returned (number 4) has relative residual 0.21. x = 0.248941606720887 0.208128063873183 0.0699707140888991 0.392812027589062 minres stopped at iteration 4 without converging to the desired tolerance 1e-06 because the maximum number of iterations was reached. The iterate returned (number 4) has relative residual 0.21. x = 4x1 GraphBLAS double matrix, sparse by col: 4 nonzeros, 4 entries (1,1) 0.248942 (2,1) 0.208128 (3,1) 0.0699707 (4,1) 0.392812
With a proper symmetric matrix
A = A+A' ; x = minres (A, b) norm (A*x-b) x = minres (GrB(A), GrB(b)) norm (A*x-b)
minres converged at iteration 4 to a solution with relative residual 1.3e-11. x = -114.061616682974 -1.42110186668133 134.822699973567 2.06940490708633 ans = 1.36498610101886e-11 minres converged at iteration 4 to a solution with relative residual 1.3e-11. x = 4x1 GraphBLAS double matrix, sparse by col: 4 nonzeros, 4 entries (1,1) -114.062 (2,1) -1.4211 (3,1) 134.823 (4,1) 2.0694 ans = 1.36498610101886e-11
Extreme performance differences between GraphBLAS and MATLAB.
The GraphBLAS operations used so far are perhaps 2x to 50x faster than the corresponding MATLAB operations, depending on how many cores your computer has. To run a demo illustrating a 500x or more speedup versus MATLAB, run this demo:
gbdemo2
It will illustrate an assignment C(I,J)=A that can take under a second in GraphBLAS but several minutes in MATLAB. To make the comparsion even more dramatic, try:
gbdemo2 (20000)
assuming you have enough memory.
Sparse logical indexing is much, much faster in GraphBLAS
The mask in GraphBLAS acts much like logical indexing in MATLAB, but it is not quite the same. MATLAB logical indexing takes the form:
C (M) = A (M)
which computes the same thing as the GraphBLAS statement:
C = GrB.assign (C, M, A)
The GrB.assign statement computes C(M)=A(M), and it is vastly faster than C(M)=A(M), even if the time to convert the GrB matrix back to a MATLAB sparse matrix is included.
GraphBLAS can also compute C (M) = A (M) using overloaded operators for subsref and subsasgn, but C = GrB.assign (C, M, A) is a bit faster.
Here are both methods in GraphBLAS (both are very fast). Setting up:
clear n = 4000 ; tic C = sprand (n, n, 0.1) ; A = 100 * sprand (n, n, 0.1) ; M = (C > 0.5) ; t_setup = toc ; fprintf ('nnz(C): %g, nnz(M): %g, nnz(A): %g\n', ... nnz(C), nnz(M), nnz(A)) ; fprintf ('\nsetup time: %g sec\n', t_setup) ;
nnz(C): 1.5226e+06, nnz(M): 761163, nnz(A): 1.52245e+06 setup time: 1.08586 sec
First method in GraphBLAS, with GrB.assign
Including the time to convert C1 from a GraphBLAS matrix to a MATLAB sparse matrix:
tic
C1 = GrB.assign (C, M, A) ;
C1 = double (C1) ;
gb_time = toc ;
fprintf ('\nGraphBLAS time: %g sec for GrB.assign\n', gb_time) ;
GraphBLAS time: 0.0875537 sec for GrB.assign
Second method in GraphBLAS, with C(M)=A(M)
now using overloaded operators, also include the time to convert back to a MATLAB sparse matrix, for good measure:
A2 = GrB (A) ;
C2 = GrB (C) ;
tic
C2 (M) = A2 (M) ;
C2 = double (C2) ;
gb_time2 = toc ;
fprintf ('\nGraphBLAS time: %g sec for C(M)=A(M)\n', gb_time2) ;
GraphBLAS time: 0.181763 sec for C(M)=A(M)
Now with MATLAB matrices, with C(M)=A(M)
Please wait, this will take about 10 minutes or so ...
tic C (M) = A (M) ; matlab_time = toc ; fprintf ('\nGraphBLAS time: %g sec (GrB.assign)\n', gb_time) ; fprintf ('\nGraphBLAS time: %g sec (overloading)\n', gb_time2) ; fprintf ('MATLAB time: %g sec\n', matlab_time) ; fprintf ('Speedup of GraphBLAS over MATLAB: %g\n', ... matlab_time / gb_time2) ; % GraphBLAS computes the exact same result with both methods: assert (isequal (C1, C)) assert (isequal (C2, C)) C1 - C C2 - C
GraphBLAS time: 0.0875537 sec (GrB.assign) GraphBLAS time: 0.181763 sec (overloading) MATLAB time: 628.349 sec Speedup of GraphBLAS over MATLAB: 3456.96 ans = All zero sparse: 4000×4000 ans = All zero sparse: 4000×4000
Limitations and their future solutions
The MATLAB interface for SuiteSparse:GraphBLAS is a work-in-progress. It has some limitations, most of which will be resolved over time.
(1) Nonblocking mode:
GraphBLAS has a 'non-blocking' mode, in which operations can be left pending and completed later. SuiteSparse:GraphBLAS uses the non-blocking mode to speed up a sequence of assignment operations, such as C(I,J)=A. However, in its MATLAB interface, this would require a MATLAB mexFunction to modify its inputs. That breaks the MATLAB API standard, so it cannot be safely done. As a result, using GraphBLAS via its MATLAB interface can be slower than when using its C API.
(2) Integer element-wise operations:
Integer operations in MATLAB saturate, so that uint8(255)+1 is 255. To allow for integer monoids, GraphBLAS uses modular arithmetic instead. This is the only way that C=A*B can be defined for integer semirings. However, saturating integer operators could be added in the future, so that element- wise integer operations on GraphBLAS sparse integer matrices could work just the same as their MATLAB counterparts.
So in the future, you could perhaps write this, for both sparse and dense integer matrices A and B:
C = GrB.eadd ('+saturate.int8', A, B)
to compute the same thing as C=A+B in MATLAB for its full int8 matrices. Note that MATLAB can do this only for dense integer matrices, since it doesn't support sparse integer matrices.
(3) Faster methods:
Most methods in this MATLAB interface are based on efficient parallel C functions in GraphBLAS itself, and are typically as fast or faster than the equivalent built-in operators and functions in MATLAB.
There are few notable exceptions; these will be addressed in the future. Dense matrices and vectors held as GraphBLAS objects are slower than their MATLAB counterparts. horzcat and vertcat, for [A B] and [A;B] when either A or B are GraphBLAS matrices, are also slow, as illustrated below in the next example.
Other methods that will be faster in the future include bandwidth, istriu, istril, isdiag, reshape, issymmetric, and ishermitian.
Here is an example that illustrates the performance of C = [A B]
clear A = sparse (rand (2000)) ; B = sparse (rand (2000)) ; tic C1 = [A B] ; matlab_time = toc ; A = GrB (A) ; B = GrB (B) ; tic C2 = [A B] ; gb_time = toc ; err = norm (C1-C2,1) fprintf ('\nMATLAB: %g sec, GraphBLAS: %g sec\n', ... matlab_time, gb_time) ; if (gb_time > matlab_time) fprintf ('GraphBLAS is slower by a factor of %g\n', ... gb_time / matlab_time) ; end
err = 0 MATLAB: 0.0693146 sec, GraphBLAS: 0.389899 sec GraphBLAS is slower by a factor of 5.62506
(4) Linear indexing:
If A is an m-by-n 2D MATLAB matrix, with n > 1, A(:) is a column vector of length m*n. The index operation A(i) accesses the ith entry in the vector A(:). This is called linear indexing in MATLAB. It is not yet available for GraphBLAS matrices in this MATLAB interface to GraphBLAS, but will be added in the future.
(5) Implicit singleton dimension expansion
In MATLAB C=A+B where A is m-by-n and B is a 1-by-n row vector implicitly expands B to a matrix, computing C(i,j)=A(i,j)+B(j). This implicit expansion is not yet suported in GraphBLAS with C=A+B. However, it can be done with C = GrB.mxm ('+.+', A, diag(GrB(B))). That's a nice example of the power of semirings, but it's not immediately obvious, and not as clear a syntax as C=A+B. The GraphBLAS/@GrB/dnn.m function uses this 'plus.plus' semiring to apply the bias to each neuron.
A = magic (3)
B = 1000:1000:3000
C1 = A + B
C2 = GrB.mxm ('+.+', A, diag (GrB (B)))
err = norm (C1-C2,1)
A = 8 1 6 3 5 7 4 9 2 B = 1000 2000 3000 C1 = 1008 2001 3006 1003 2005 3007 1004 2009 3002 C2 = 3x3 GraphBLAS double matrix, sparse by col: 9 nonzeros, 9 entries (1,1) 1008 (2,1) 1003 (3,1) 1004 (1,2) 2001 (2,2) 2005 (3,2) 2009 (1,3) 3006 (2,3) 3007 (3,3) 3002 err = 0
(6) Performance issues
The GrB matrix is a MATLAB object, and there are some cases where performance issues can arise. Extracting the contents of a MATLAB object (G.field) takes much more time than for a MATLAB struct with the same syntax, and building an object has similar issues. The difference is small, and it does not affect large problems. But if you have many calls to GrB operations with a small amount of work, then the time can be dominated by the MATLAB object-oriented overhead.
A = rand (3,4) ; G = GrB (A) ; tic for k = 1:100000 [m, n] = size (A) ; end toc tic for k = 1:100000 [m, n] = size (G) ; end toc
Elapsed time is 0.052089 seconds. Elapsed time is 0.524830 seconds.
GraphBLAS operations
In addition to the overloaded operators (such as C=A*B) and overloaded functions (such as L=tril(A)), GraphBLAS also has methods of the form GrB.method. Most of them take an optional input matrix Cin, which is the initial value of the matrix C for the expression below, an optional mask matrix M, and an optional accumulator operator.
C<#M,replace> = accum (C, T)
In the above expression, #M is either empty (no mask), M (with a mask matrix) or ~M (with a complemented mask matrix), as determined by the descriptor. 'replace' can be used to clear C after it is used in accum(C,T) but before it is assigned with C<...> = Z, where Z=accum(C,T). The matrix T is the result of some operation, such as T=A*B for GrB.mxm, or T=op(A,B) for GrB.eadd.
For a complete list of GraphBLAS overloaded operators and methods, type:
help GrB
Thanks for watching!
Tim Davis, Texas A&M University, http://faculty.cse.tamu.edu/davis See also doc sparse and https://twitter.com/DocSparse