We will establish a connection between operator-valued infinitesimal freeness and freeness with amalgamation over subalgebra. Then, we will introduce the operator-valued infinitesimal cumulants and some related properties. It can be applied to build the relation between scalar-valued infinitesimal freeness and Matrix-valued infinitesimal freeness. Finally, we will show how to construct the operator-valued infinitesimal free additive convolution.