According to this article, quote marks are escaped in a variety of ways.
Here are some examples coming from the aforementioned article:
'Mike\'s House'
"Mike's House"
'Mike''s House'
From the above examples, only the double single-quote character escaping works in SQL CE:
The other two queries are invalid and cause parsing errors. Supporting only this case is sufficient.