This is a semi-legendary story with shifty details told to showcase Frederick's enlightened ways with Muslims and intellectual prowess. There might be some historical basis to it, although Muslim historians had reasons to exaggerate and embellish considering how friendly Frederick was compared to other Christian crusaders. In any case, the original sources give no details on the content of the problems.
According to Ibn-Wasil, a contemporary chronicler, the problems were sent as a challenge to ease tensions during negotiations over the status of Jerusalem during Frederick's siege of it, see Two sources on the handover of Jerusalem to Frederick II:
"The emir Fakhr al-Din ibn al-Shaykh carried the messages between him and the king-emperor, and between them there were discussions on various topics. During these the emperor sent al-Malik al-Kamil difficult philosophical, geometrical and mathematical questions, in order to put the learned men who were with him to the test. Al-Malik al-Kamil showed the mathematical questions that had come to him to the shaykh ‘Alam al-Din Qaysar ibn Abi’l-Qasim, who was a leader in the field, and showed the remainder to a group of learned men, and they answered the lot. Then the sultan al-Malik al-Kamil swore to abide by what they had agreed on, as did the emperor, and they concluded a peace agreement for a fixed term."
A similar account is given by Maqrīzī, see Frederick II's crusade by Takayama. Unfortunately, neither of them gives any details on the problems. Alam al-Din Qaysar al-Katib al-Hanafi, nicknamed Ta'asif, is an Egyptian born scholar of the time who designed a celestial globe that al-Kamil sent to Frederick as a diplomatic gift c. 1225. He also wrote a geometric treatise on Euclid's postulates, but is better known for engineering feats and astronomy. Architects and Artists in Mamluk Society by Rabbat describes him as follows:
"He was a katib (a scribe or clerk serving in the diwan, or chancery, in Cairo), a faqih (jurist, though not a refined one), a geometrician, mathematician, and astronomer, and a muhandis, which seems here to mean a civil and military engineer. He is credited with building a mill on the Orontes for the Ayyubid king of Hama, al-Muzzafar Mahmud (d. 1244), and an unspecified number of towers around the city, for which he invented a number of engineering devices, or perhaps designs (hiyal handasiyya). He is also said to have built a wooden globe for the same ruler on which he marked all the stars in the celestial sphere (preserved at the Museo Nazionale in Naples)."
There also exist multiple other instances where Frederick is described as having sent questions on various subjects to Muslim scholars. One source with more details is the well-known work of Ibn-Sab'in, Sicilian Questions, where the answers to Frederick's questions are supposedly given. But Ibn-Sab'in only answers philosophical ones (on the eternity of the world, soul, and the like), and Frederick's involvement is most likely his fabrication, see Weltecke, Emperor Frederick II:
"One of the sources Abulafia relies on is a famous Muslim text reporting on Frederick sending philosophical questions to the Arab world. These so called Sicilian Questions had always been a classical topos in the Orientalist construction. They were regarded as evidence for Frederick's close contacts to Muslim scholars and his interest in intricate philosophical problems...
At the time nobody could anticipate that Anna Akasoy would today very
convincingly suggest that the Sicilian Questions were not authentic: Frederick
could not have been acquainted with Ibn Sab'in because at the time the latter
was not a known scholar at all. In fact, Ibn Sab'in was in his early twenties when he wrote the Sicilian Questions, which was his first book. It is therefore highly unlikely that Frederick sent him these philosophical questions and, likewise, that the Arab answered him in this fashion in a personal letter. Instead, Ibn Sab'in invented the questions himself and probably used the emperor's famous name and anti-Christian slander to sell his book in Morocco".