Re: Romanizing arabic text - long post with arabic - ASP.NET Discussion Boards

I have copied all dlls from windows folders to my website bin folder, and it worked.

Nice that it had worked copying the dll's, but this solution of moving the dll's didnt work for me

Naina

Salam guys;
I am trying to write a tool that is supposed to romanize arabic text. Do you now where i can get an algorithm or any thing useful.

Help people,so poeple can help you.

I think the biggest problem is caused by the "vocals" which could also be consonants. Look at your name: it starts with a waf, doesn't it, and hence could also be transliterated to Wamairi or Umairi; likewise with the yod - Omayari or Omayiri could also be possible. You must find some rules for that, otherwise you can only use a "look-up table", i.e. a Dictionary<string, string=""> containing pairs of Arabic and romanized versions for every word.
And when it comes to other languages using the Arabic alphabet (e.g. Persian, Pashtu, Urdu, traditional Malay, Turkish before Atatürk), the situation becomes even more complex.

Sir;
You are right the hardest job is to determine what the word is (hence the data entry wouldnt enter the word with vocals, with addition to hamzah, maddah, and shaddah which may be emitted). And the reset will be easy.

by the way, you know much about arabic (where did you learn that?!!) Wink | ;)

Help people,so poeple can help you.

Can't see why your question was downvoted, it seems a reasonable question, even if the answer is not forthcoming.

------------------------------------

I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave

CCC Link[^]
Trolls[^]

i think its because when i first come here i was an annoying person and i made some enemies.
they might not understand the question but they judge the question by what they feel about me.
this is what most people do Smile | :)

Help people,so poeple can help you.

I got your message. Though I don't mind people e-mailing via CP, it would be much better to leave a message as the link gets lost & I need to do a search.

First off, I can [mostly] read the script, but I can't understand what has been written unless it is very simple so, mod everything I say through this knowledge. You might want to look up "transcription" - this is the proper term for what you are trying to do (i.e. transcription from Arabic to English), there may be good algorithms to do this out there, google has one in the opposite direction.

I'd start by mapping various letters to their latin equivalent in a dictionary ا =>a ب=> b etc. The problems start when there is no direct Latin equivalent. Some have well-known tranlisterations(ع => a',غ => gh خ=> kh) but will not be pronounced properly by an English reader for the most part. The other immediate problem you have are the "doubled" up letters: ق/ك normally transliterate to k/q but the q isn't really hard enough and will be pronounced as k by Europeans. Worse are the parings س/ص د/ض and the various "th" and "h" sounds. When I studied I used capital letters to denote the heavier sounds, but this will look odd. If you don't capitalize the Arabic will be incorrect, I think using the capitals will be better. All of these are surmountable with some flexibility.

Your real problems start with vowels. the distinction between اع (a'a)and عا (aa') is pretty fine and is confused by the fact that both letters can be sounded aa/a/u/i depending on the diacritic marks. ئ andء confuse the issue further. Next up is ي which can be sounded y or ee (and is normally transliterated "i" as in habibi). I don't know how to work round this problem (and وو which can be woo as in dawoud or oo ) probably they can be worked out algorithmically from the preceding letters, with the ability to override from a dictionary for quirks (a similar issue arises from ة and its position in the sentence). The final thing is text without hamza, sukkun damma, kasra etc. Without these diacritics, the whole thing becomes a research project, for example موقف => mwqf (resulting in a WTF Laugh | :laugh:

) instead of muwaqaf.

My advice:

Start with vowel/punctuated/diacritic marked text
Find a standard and stick to it, wikipedia has a page about Romanisation from Arabic[^]. One readers are used to your standard it will become easier to them
pick of the easy stuff in a character==>character dictionary
Work on algorithms to figure out the harder stuff

This last point, the best way is to throw real text and get someone to read the results. If you want un-vowelled text you'll need some way to cope, I've heard of working it out from the three-letter root and grammatical position, but this is a complex problem (one of the academics at work specializes in this field). Finally, the results can't be perfect: your users will just need to be aware of this, in the Seven Pillars of Wisdom by TE Lawrence, he deliberately transcribes names differently in different places. In the preface he admits this is partly to upset the proof readers, but partly to make the point that it can't be perfectly Latinized.

Sort of a cross between Lawrence of Arabia and Dilbert.[^]
-Or-
A Dead ringer for Kate Winslett[^]

Sir;
at first thanks for your message. about the e-mail I'm sorry, but there is no 'Replay' link under your message 'Please leave message after the tone' on your board[^].

Sir;
actually, I am done with the first three points, and I am working on the fourth one(the algorithm). It should take a partially marked arabic word and returns several possibilities of the fully marked version. But, my algorithm is still too slow and it comes with too many possibilities, Also I can't handle the prefixes added to nouns(like al-(noun), wa-(noun), bi-(noun), fa-(noun), lil-(noun), ..etc).

Do you know how i can fix these?

Help people,so poeple can help you.

I'm sorry but I can't help, it is beyond my capabilities. I attended a seminar about this topic and the same problem was highlighted: too many results for the same letter groups. The approach to solve the problem was heuristic, taking the context of the sentence and matching / restricting the results from that. There is a full research project between my university and a company who provide searches in Arabic (they don't translate, but the problem domain of getting the full word from the text is the same). One problem is the sheer processing power needed for this. IMO there are two possibilities for the heuristic analysis, a bayesian beleif network is probably a good option and the most obviously applicable.I also thought a neural network might also be trained to perform it should also get better the more it is used.
Sorry to be so negative, but this is a hard problem, interesting though!

Sort of a cross between Lawrence of Arabia and Dilbert.[^]
-Or-
A Dead ringer for Kate Winslett[^]

Hi Sir;
Finally, I made the romanization tool. Actually, the problem wasn't the romanization process itself but the word to be romanized.

First my tool accepts a text from the user and parse it into a collection of arabic words and separates(strings with no arabic letters). Then each word would be checked against a dictionary in the database. If if doesn't exist, the words would passed through an algorithm to suggest a fully adjustment for them according to specific rules. And after displaying the results the user of my tool can edit the romanized text in a modal dialog by choosing the correct adjustment of each word and then the roomanized version will be changed accordingly.
Finally, on accepting the dialog the results will be saved to the dictionary in the database.

Help people,so poeple can help you.

14 Months between my reply and yours, I said it would be hard LOL!

I'm glad you got it working. Thumbs Up | :thumbsup:

Sort of a cross between Lawrence of Arabia and Dilbert.[^]
-Or-
A Dead ringer for Kate Winslett[^]

Actually, I Developed two versions of the tool; one for desktop and the other for web(as an extender). And I was stopped several times because the requirements were not clear. In addition, enhancing of the algorithm took 6 months.

between you and me, it was a long exciting journey and i learned alot from that.

Help people,so poeple can help you.

I want to add a Company field to my forms authentication so that when a user logs in I can determine what company he belongs to and constrain the records he or she sees accordingly. Can someone point me to a good tutorial for this? I have no problem writing the code for it, I just need something that will point me in the right direction with a minimum (I hope) amount of fuss.

Cheers, --EA

If you are using the ASP.NET Login control use the LayoutTemplate to add your controls then handle the LoggingIn event to validate.

Not terribly difficult to find information on this
ASP.NET Login Controls Overview
[^]
Customizing the Appearance of ASP.NET Login Controls
[^]

No comment

I should have probably mentioned that I am using the MVC3 toolkit which has it's own built in account view and controller. I am sure I can dig into the code and add the parts I need as I have access to the tables and the code, I just thought there might be an easier way without using a login control.

eddieangel wrote:
I should have probably mentioned that I am using the MVC3 toolkit

Yes, a fairly significant ommission.

The basic concept is the same though, just different framework.

No comment

It might have been easier for me to ask if there was a way within the current ASP.net forms authentication model to store the users company name and access that information as needed. I don't need to reinvent the wheel, I just need to see what company the user belongs to when he is viewing data.

You don't need to reinvent the wheel. Microsoft has done good job of providing for extensibility. The Login control can use templates and the authentication mechanism uses the Provide pattern. Use the template to add your controls and create a Forms Authentication Provider[^] to handle your needs.

No comment

Question is described by the steps below.
1.Create a new ASP.NET MVC3 Web Application using Internet Application Template.
2.Decorate HomeController class with Authorize.
3.Run the application, It basically takes you http://localhost: 51578/Account/LogOn?ReturnUrl=%2F Everything good so far.

Let’s customize a couple of things in this project.
1.Add an area named “Login” in this solution and move AccountController.cs, AccountModel.cs and whole Account folder under Viewer into appropriate folder under this Area named Login.
2.Run the application now and you will get following error

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: The view 'LogOn' or its master was not found or no view engine supports the searched locations. The following locations were searched: ~/Views/Account/LogOn.aspx
~/Views/Account/LogOn.ascx
~/Views/Shared/LogOn.aspx
~/Views/Shared/LogOn.ascx
~/Views/Account/LogOn.cshtml
~/Views/Account/LogOn.vbhtml
~/Views/Shared/LogOn.cshtml
~/Views/Shared/LogOn.vbhtml

Do I need to modify anything in the web.config or in routing to make this work?

I am not sure if you have to change the routing or not, but I know that when I am working with a view outside the scope of the current controller I specify it as a third parameter in the link or redirect.

Ex:

@Html.ActionLink("Back to Claims", "Index", "Claim")

Where arg 1 is the name of the link, arg 2 is the action and arg 3 is the appropriate controller.

Also, if you look at the web.config file there is a forms loginUrl attribute that you can change to point to the correct login page, I imagine.

Cheers, --EA

I have tried changing the path in web.config for loginUrl attribute but didn't had any luck. It just keep complaining about access issues whenever I direct it outside ~\View folder.

As for @Html.ActionLink, I am not sure where to force it. Because re-direction to Logon page is happening due to fact that I have decorated my HomeController with Authorize attribute and therefore there is no @Html.ActionLink involved.

I think that from an organizational standpoint the issue is that the top level folder structure should be View - Model - Controller. Build your application and see where it places the account view that you are looking for, whether you put it somewhere else or not in the folder structure for Visual studio, when it compiles up it should end up under the deployed views folder, I think.

I don't know for sure though, look and see where the deployed page lands in the folder structure in relation to the "views" folder, which is considered the "top level" folder for your web pages.

Edit: Check out this link in regards to registering areas. For me it all seems a little over complicated, I prefer the classic Model - View - Controller structure with each of those items having relevant subfolders (Model - Account, View - Account, Controller - Account)

I am not changing any organizational strucutre.If you follow my instructions from original question, everything is wherever prjoect create by default. All I am doing is that I am creating a new area, but not any custom location. Its wherever VS created by default. The only thing is that its been called by framework itself as I am just applying an attribute and not any Html.XXXX() call.

I think if you add an area you are going to have to mess around with routes. I don't know because I haven't created any areas, I just use the default View, Model and Controller folders created when you create a project using the MVC3 template in VS2010 without adding an area. To me it is still clean because when you want to generate a controller, you just generate it under the controllers folder. When you generate that controller with scaffolding it creates all of the views in a clean folder under the views folder.

Just me though, if the application is going to get massive and unwieldy, maybe that is a good way to do it, but I haven't read enough about areas to see an appeal.

Sorry I couldn't be more help.