There is a minor point in David's answer, which is that not all index collisions are taken care of in how the g matrix is constructed...
So the "sparse" method I suggested, which is basically the same in spirit as David's answer except it doesn't need sub2ind and assumes we want sparse, will give a different answer because instead of overwriting repeated index (as both above methods do), using sparse with repeated subscripts will add the values that are repeated.
So slightly modifying the original loop version to be additive (I know it changes the rules, but just to keep this answer simpler), you can compare timing. On my computer the sparse method is faster as m increases. I observe that David's vectorized answer does see some speed gain versus looping, but advantage scales differently than the sparse method...I'm speculating now, but I think that's because the bottle neck is indexing into full matrices...but i'm no computer scientist so I'll let better minds than mine chime in on that.
irep = (g(1,:)-g(2,:))==0;
A(g(1:2,k),g(1:2,k)) = A(g(1:2,k),g(1:2,k)) + g(3,k)*[1,2;3 4];
sub1 = [ga(1,:); ga; ga(2,:)];
A2 = sparse(sub1,sub2,[1 2 3 4]'*gb,N,N);
err = max(abs(A-A2),,'all')
A better apples-to-apples comparison would be if the matrix g was constructed such that all index collisions are avoided, or if the original application doesn't prohibit that, need some clarification on intended behavior.