There are already some good examples here of words beginning with y and z.  X is a trickier matter, and you would have to make a serious case for any word you chose to fit this category in English.  Most words that begin with x in the English language are of Greek origin and/or were added to the language centuries after the Renaissance period.   “Xenophobia,” for example, is an early 20th century construction, and “x-ray” didn’t appear until the mid-1800s.

By a small stretch you could use “Xmas” for this entry; it’s first known use was in 1551, X being visually of the same form as the Greek letter chi, the first letter in Christos, the Greek word for Christ.

There is also written evidence for the use of the letter x- rather than sh- or sch- at the beginnings of words in East Anglian, though this was in the fourteenth century.  For example the word shall was instead written xall.  This was not a widespread practice, however, and is an even further stretch than “Xmas.”