Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with text editor find/replace and utf-8 characters. #323

Open
paxcut opened this issue Dec 18, 2023 · 0 comments
Open

Problems with text editor find/replace and utf-8 characters. #323

paxcut opened this issue Dec 18, 2023 · 0 comments

Comments

@paxcut
Copy link

paxcut commented Dec 18, 2023

First let me say that I really like SHADERed. Thank you for sharing it with everyone.
I noticed that you implemented find/replace in the text editor. Another app that uses a fork of the text editor is ImHex which didn't have find/replace support. It will soon because thanks to your code I was able to implement the much needed feature. Since the editor support utf-8 encodings in original branch, then I wanted to make sure that find/replace worked well with that particular Unicode encoding. As it stands, however, including utf-8 characters in the shader programs breaks search/replace.

There are two issues to address here. First is the fact that mFindWord (the searched text) may contain utf-8 chars itself so you can't use strlen which gives you the number of bytes to calculate its length in chars. I fixed that by creating a function similar to GetLineCharacterCount that works on strings instead. The second issue is a bit harder to fix because it involves the fact that std::string based std::find returns the number of bytes you have to skip to find the target sub-string and not the number of characters (ascii or otherwise) that exist before the target.

What this means is that the variable called cindex that is used to locate the target using the result of calling std::find is not using the same units needed (in this case bytes) in order to combine it with result of find which also uses bytes. cindex is using units of chars instead so the selections the find produces are in the wrong place. In practice all you need to do to resolve this is replace the calls to GetLineCharacterCount with calls to a GetLineByteCount function that returns the strlen of the lines in bytes instead of counting chars.

Doing both changes fixed the find/replace functionality in the presence of utf-8 encoded chars in ImHex and they should fix it in SHADERed too. I wanted to make sure I shared the information with you in case it proves to be useful. If my explanation is confusing I can post code or even submit a PR whichever you prefer.

Thank you again for your time and effort.

PS: I wanted to post this in the text editor fork but issues are disabled there, I hope it is not too off-topic to post here but this will affect the use of SHADERed too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant