the vector api

the picture

a checkout with one cashier scanning one item at a time, versus a cashier with a scanner wide enough to read four items in one beep. the work per item is the same; the number of beeps drops by four. a 128-bit register holds four 32-bit floats, so one instruction does four multiply-adds. a 512-bit one (AVX-512 on some x86 servers) does sixteen.

the code

the element-wise loop, both ways. the vector version processes SPECIES.length() elements per iteration, then mops up the tail:

import jdk.incubator.vector.*;
 
static final VectorSpecies<Float> SPECIES = FloatVector.SPECIES_PREFERRED;
 
static void scalar(float[] a, float[] b, float[] c) {
    for (int i = 0; i < a.length; i++) c[i] = a[i] * b[i] + c[i];
}
 
static void vector(float[] a, float[] b, float[] c) {
    int i = 0;
    int upper = SPECIES.loopBound(a.length);
    for (; i < upper; i += SPECIES.length()) {
        FloatVector va = FloatVector.fromArray(SPECIES, a, i);
        FloatVector vb = FloatVector.fromArray(SPECIES, b, i);
        FloatVector vc = FloatVector.fromArray(SPECIES, c, i);
        va.fma(vb, vc).intoArray(c, i);              // c = a * b + c, one lane per element
    }
    for (; i < a.length; i++) c[i] = a[i] * b[i] + c[i];   // the tail that doesn't fill a vector
}
 
void main() {
    IO.println(SPECIES + ": " + SPECIES.length() + " lanes of float, " + SPECIES.vectorBitSize() + " bits");
    float[] a = new float[10], b = new float[10], c1 = new float[10], c2 = new float[10];
    for (int i = 0; i < 10; i++) { a[i] = i; b[i] = 2; c1[i] = c2[i] = 1; }
    scalar(a, b, c1);
    vector(a, b, c2);
    IO.println(java.util.Arrays.toString(c1));
    IO.println(java.util.Arrays.toString(c2));
}

an incubator module isn't resolved by default, so it needs a flag at compile time and at run time (which is also why there's no run button here):

$ java --add-modules jdk.incubator.vector Main.java
WARNING: Using incubator modules: jdk.incubator.vector
Species[float, 4, S_128_BIT]: 4 lanes of float, 128 bits
[1.0, 3.0, 5.0, 7.0, 9.0, 11.0, 13.0, 15.0, 17.0, 19.0]
[1.0, 3.0, 5.0, 7.0, 9.0, 11.0, 13.0, 15.0, 17.0, 19.0]

the pieces: a species is element type plus shape (float × 128 bits = 4 lanes). SPECIES_PREFERRED asks for the widest shape this CPU handles well, so the same code uses 4 lanes here and 8 or 16 on an AVX2 or AVX-512 machine. fromArray/intoArray load and store whole registers, fma is a fused multiply-add per lane, and loopBound rounds the length down to a multiple of the lane count. there are lane-wise operations for everything arithmetic, comparisons that produce masks, masked loads for tails, shuffles, and reductions that combine all lanes into one value.

does it actually help?

measured with JMH on the M4 Pro, 1024 floats, scalar vs Vector API:

that's the honest story of the Vector API. C2 has had an auto-vectorizer (SuperWord) for years: simple element-wise loops over arrays, like the first one, already turn into SIMD instructions with no help, and here the hand-written version was slightly slower.

but the dot product (sum += a[i] * b[i]) is a reduction, and floating-point addition isn't associative: (x + y) + z can differ from x + (y + z) in the last bits (floating point). adding four partial sums in parallel changes the order of the additions, so the JIT isn't allowed to do it for you; it has to add one element at a time, each addition waiting on the last. reduceLanes(ADD) is you saying "i accept a different rounding order", and then four lanes run in parallel. same for code the auto-vectorizer just doesn't recognize: branches turned into masks, lookups through shuffles, mixed types.

why it's still incubating

the API was ready years ago. what it's waiting for is valhalla. a FloatVector is a value: it has no meaningful identity, you never want to compare two with == or lock on one, and the JIT tries hard to keep it in a register and never allocate it. today it's an ordinary class marked value-based, and when the JIT can't keep a vector in a register (it escapes, or the code isn't compiled yet), it really does allocate an object per vector, and performance collapses. the plan in the JEP is to turn Vector and its subclasses into value classes once valhalla's are available as a preview, move the API from incubator to preview, and then finalize it.

incubator status has practical consequences: the package is jdk.incubator.vector, it needs --add-modules everywhere, it prints that warning at startup, and the API can still change between releases (it's had small changes along the way). libraries that want SIMD today mostly use it behind a runtime check (Lucene does this for vector search), or rely on the auto-vectorizer.