With the recent rising application of mathematical models in the field of computational systems biology, the interest in sensitivity analysis methods had increased. The stochastic approach, based on chemical master equations, and the deterministic approach, based on ordinary differential equations (ODEs), are the two main approaches for analyzing mathematical models of biochemical systems. In this work, the performance of these approaches to compute sensitivity coefficients is explored in situations where stochastic and deterministic simulation can potentially provide different results (systems with unstable steady states, oscillators with population extinction and bistable systems). We consider two methods in the deterministic approach, namely the direct differential method and the finite difference method, and five methods in the stochastic approach, namely the Girsanov transformation, the independent random number method, the common random number method, the coupled finite difference method and the rejection-based finite difference method. The reviewed methods are compared in terms of sensitivity values and computational time to identify differences in outcome that can highlight conditions in which one approach performs better than the other.