check that you only calculate what you need and extract ANY constant value out of the loops and use const where possible.
Like in
for(int x = J / 4 * 4; x < J; x++){
result->data[i * J + x] += mat1 -> data[i * K + k] * mat2 -> data[k * J + x];
}
the constant
int J4 = J / 4 * 4;
matrix *mr = result->data[i * J];
const matrix *m1 = mat1->data[i * K + k];
const matrix *m2 = mat2 -> data[k * J};
for(int x = J4; x < J; x++){
mr[x] += m1 * mat2->m3[x];
}
Advanced is to
create the assembly code and look if it shows some strange operations like type conversion or data aligments.