Bitcoin would not work if it weren't for the cryptographic algorithms it uses. The main cryptographic components used in Bitcoin are keys, addresses and signatures.
A bitcoin address is similar to a bank account number for conventional currencies. It might look something like this
BM1Ntn2onLAYoqZVFsLM8qpThCDV33i9gT
Bitcoin uses an asymmetric encryption algorithm known as ECDSA (Elliptic Curve Digital Signature Algorithm). Asymmetric encryption means that users have a pair of keys, one to encrypt data, and another one to decrypt data. The encryption key is called the public key and the decryption key is called the private key. Users can share their public key so that anyone can encrypt data for them, but the private key must be kept secret so that only they can decrypt data.
In contrast, a symmetric encryption algorithm is one that uses the same key for encryption and decryption.
To create a key pair we start by generating a private key. The private key is simply a number $p$ between $0$ and $2^256$ that needs to be generated. We could do it by flipping a coin 256 times, but modern operating systems have modules for pseudo-random number generation. The number needs to be random enough so it can't be guessed by anyone else.
The full theory behind elliptic curve cryptography is very mathematical and requires a good understanding of number theory. Here is the short version.
An elliptic curve is defined by the following equation where $a$ and $b$ are real numbers
$$y^2 = x^3 + ax + b$$
We define the elliptic curve over a finite field with the following addition operation. To add two points $P_1$ and $P_2$ on the curve, we take the line between them (or the tangent line if $P_1=P_2$) and find the third point of intersection (there will always be one) then the sum $P_3$ is that point mirrored over the x-axis.
Multiplication with an integer is then defined in the usual way
$$k*P = P+P+\dots+P$$
Bitcoin uses an elliptic curve known as secp256k1 as defined by NIST.
When we have generated a large random number to use as private key $m$, we derive the public key $M$ by multiplying it with a generator point $G$ as defined above, $M = m*G$. We then have a key pair $(m,M)$ that we can use for our Bitcoin transactions.