Stability of an elastic column with a changeable cross section loaded by a concentrated force of arbitrary intensity is studied. It is assumed that the column is fixed to a rigid circular plate that is positioned on a homogeneous, isotropic, linearly elastic half-space. The constitutive equation of a column allows effects of axial compressibility and shear stresses. The bifurcation points of the full nonlinear system are determined by the eigenvalues of the linearized equations. It is shown that different types of bifurcations dependent on values of parameters can occur. The post critical shape of the column is determined by numerical integration of the system of equilibrium equations.