Perfectly inelastic supply means that suppliers will provide the same amount of product regardless of the price. In this case, if a new sales tax is imposed on the product, the seller will be willing to take the whole burden of the tax. Please note that the seller will not WANT to do this and WILL pass the cost of the tax on to the consumer if possible.
However, if the consumers will not bear the burden of the tax, the seller will because, to the seller, the price of the product (or how much the seller is getting for it) is irrelevant -- they're going to provide it no matter what the price is.
A elasticities of demand and supply are calculated separately for each given level of price. Thus elasticity of supply is defined as ratio of percentage change in demand to corresponding change in supply. It is possible to conceive situation when the supply of a commodity by a particular supplier will be completely inelastic within a given price range, that is, there will be no change in quantity offered for supply for changes in prices within that range. But, as per principles of economics, it is not possible that quantity supplied will remain same irrespective of the prices. Thus we cannot expect a firm to remain in business if people will accept its product only when given free.
Coming to the question of paying tax, a supplier pays tax only when he is required to pay it as per tax laws and provision. The only effect of tax is to lower the effective price realization of the supplier. However, when the supply is assumed to be inelastic, the supplier will continue to offer same quantity for supply irrespective of the tax rate or presence or absence of tax.