Every quantity has some SI units like distance, time, speed etc. Why do we prefer SI units for these quantities?
What is the importance of SI units in Physics?
If by "physics" you mean units used by professional physicists and related disciplines, it's not as much as one would think.
While motivated by science, SI units exist primarily to benefit commerce, not science. The ordinary, everyday phenomena described well in terms of SI units aren't the phenomena that physicists currently study. (They used to study those phenomena, but that was long ago. Physicists have moved on to studying much, much smaller and much, much bigger things.
A widely used system is theoretical physics is Planck units, where the gravitational constant $G$, the speed of light $c$, the reduced Planck constant $\hbar=h/(2\pi)$, the Coulomb constant $k_e = 1/(4\pi\varepsilon_0)$, and the Boltzmann constant $k_B$ all have numerical values of one. These choices lead to rather different values for the unit of length, unit of time, unit of mass, and unit of charge than in SI. The relationship isn't all that precise because the SI value of $G$ is only good to four decimal places of accuracy. That problem disappears in Planck units, where $G$ is exactly one.
Astronomers and astrophysics who study very remote phenomena use redshift as their measure of time. They might translate this to years, but that requires a specific model of the expansion of the universe. Redshift is observable. The time at which some phenomenon that occurred long ago is not. What about distance? Distance is even more problematic. In their professional writing (i.e. peer-reviewed journal papers), you'll rarely see distance.
Astronomers and astrophysicists who study not quite so remote phenomena do use distance, but their unit of choice is the parsec. The issue of expansion of space is less problematic for something observed a few galaxies away. The unit of time to these astronomers and physicists is the Julian year, or 365.25 $\cdot$ 86400 SI seconds. The unit of mass is the solar mass. Astronomers and astrophysicists who study solar system bodies also use the solar mass as the unit mass, but the unit of distance used by these scientists is the astronomical unit, and the unit of time is typically a day (but sometimes a Julian year). The large uncertainty in $G$ once again gets in the way of translating to/from SI units.
At a much, much smaller scale, particle physicists working at the atomic scale oftentimes use the electron charge and electron mass as the units of charge and mass; the other units are chosen to make $\hbar$, $k_e$, and $k_B$ all have numerical values of one. Nuclear physicists tend to make $c$, $\hbar$, and $k_B$ have numerical values of one. Some nuclear physicists use the proton mass as the unit of mass, others use the electron-volt as the unit of mass-energy (since $c=\hbar=k_B=1$, energy, mass, and temperature all have the same dimensions).
The Système International d'Unités (SI) is not just a collection of units but rather a system of units*. The units all work together, so if you use some quantities in SI units, other quantities that you calculate from them will also be in SI units.
You give the examples "distance, time, speed etc." These units are related: if you give distance in meters (m) and time in seconds (s), the calculated speed found by dividing these units will be given in the SI unite meters per second (m/s). If you had an object and measured its mass in kilograms (kg) and calculated work or energy, you would get the SI unit Joule (J) which is equivalent to kg m^2/s^2. You know in advance just which unit you will get, with no possibility of getting calories, Calories, BTU's, or ergs.
There are other systems, such as cgs (centimeters, grams, seconds), and these are useful in some situations. But at the human-sized level, SI measures well most things that we work with. (The main exception is volume, where m^3 is too much for common use.)
This deals with which particular metric system is used. Other, non-metric systems such as Imperial (or Avoirdupois or American customary units), are best dealt with separately. Suffice it to say that almost every country in the world has settled on metric systems.
SI units are important because:
They are common to the people of the entire world, so that people from different countries can communicate with each other conveniently regarding business and science.
It makes systematic use of prefixes, making it easy to express very large or very small numbers.
It makes calculations very fast.
SI units are interrelated in such a way that one unit is derived from other units without conversion factors. SI is used in most places around the world, so our use of it allows scientists from disparate regions to use a single standard in communicating scientific data without vocabulary confusion. SI units are important because these are common to the people of the entire world, so that people from different countries can communicate with each other conveniently regarding business and science. It makes systematic use of prefixes, making it easy to express very large or very small numbers.