You can use this kind of calculator for any kind test that has a true or false type of conversion. That could include testing email subject lines, CTA text or parts of your registration flow. You either opened the email, or you didn't. You either clicked the button, or you didn't. You either registered, or you didn't/. This type of conversion-based calculator does not work for tests where you are, for example, trying to increase a metric in an 'average by user' sense.
You would not use this kind of calculator if you're trying to determine an average increase in the amount of time users spend on your site, or an average increase in cart size / spend in your checkout. For that kind of test, you need to factor in the conversion rate of each visitor.
The confidence refers to the uncertainty of the results in an A/B test. Imagine you run a test and there is a 10% lift in conversions. Statistically speaking, you should not expect the exact same results each time. That uncertainty is represented in this calculator as the standard error of the sample (SE A and SE B). With a 95% confidence interval, you have a 5% chance of being wrong when you decide to implement the winning variation of your test.
This calculator will show you whether the test has reach a 90, 95 or 99 percent confidence interval. If you're running a test and you want to be really sure you make the right choice, just let it run longer! A larger sample size will yield more confidence in the results.
You can either let the test keep running until you do reach significance or you can use a Bayesian A/B testing calculator (google it) that gives you the probability that your test variation beats the control.