Contrary to what is commonly thought, it is possible to obtain convergent results with negative (rather than positive) penalty functions. This has been shown and proven on various occasions for vibration analysis, but in this contribution it will also be shown and proven for systems of linear equations subjected to one or more constraints. As a key ingredient in the developed arguments, a pseudo-force is identified as the derivative of the constrained degree of freedom with respect to the inverse of the penalty parameter. Since this pseudo-force can be proven to be constant for large absolute values of the penalty parameter, it follows that the exact solution is bounded by the results obtained with negative and positive penalty parameters. The mathematical proofs are presented and two examples are shown to illustrate the principles.