|
I an having some trouble with removing duplicate entries from an xml file.
I am using mostly Linq to XML and C# to build the list.So I would like a Linq to Xml aproach further too.
Example(before):
<SYNSET>
<ID>new_id</ID>
<SYNONYM>
<LITERAL>word1<SENSE>II.♦</SENSE></LITERAL>
</SYNONYM>
<DEF> definition1 </DEF>
</SYNSET>
<SYNSET>
<ID>new_id</ID>
<SYNONYM>
<LITERAL>word2<SENSE>I</SENSE></LITERAL>
</SYNONYM>
<DEF> definition2 </DEF>
</SYNSET>
<SYNSET>
<ID>new_id</ID>
<SYNONYM>
<LITERAL>word1<SENSE>II.♦</SENSE></LITERAL>
</SYNONYM>
<DEF> definition1 </DEF>
</SYNSET>
After it should be:
<SYNSET>
<ID>new_id</ID>
<SYNONYM>
<LITERAL>word1<SENSE>II.♦</SENSE></LITERAL>
</SYNONYM>
<DEF> definition1 </DEF>
</SYNSET>
<SYNSET>
<ID>new_id</ID>
<SYNONYM>
<LITERAL>word2<SENSE>I</SENSE></LITERAL>
</SYNONYM>
<DEF> definition2 </DEF>
</SYNSET>
The xml database should have around 100k entries like this, so I need a fast method to remove duplicates. Thanks in advance
|
|
|
|
|
Probably be best to use a hash table...
have a look at the Dictionary[^] class.
Then read you xml file.
Loop all the entries.
Build a string[^] of all entry values.
Create a hash[^] of that string.
Check if the hash value is in the dictionary (with its Contains function)
If its not there then add it to the dictionary then export the entry row to the new xml file.
End of loop.
If you skip the hash step and simply add the built string then it will use more memory but the process speed may be faster. You will have to test and decide yourself.
Life goes very fast. Tomorrow, today is already yesterday.
|
|
|
|
|
I dont think it will really work. As I said to you the xml database has around 100.000 entries. What I wanna do is check if an entry appears 2 times in that database and remove the duplicates. I read the dictionary class a bit, not really what I am looking for.
|
|
|
|
|
Well I have used the same process to remove duplicate lines from a text file (100MB+) so I know it will work.
How do you expect to check if a line is a duplicate? you have to check each line against all other lines in the file right?
you could skip the dictionary part and check all lines in the new file each time you read one in from the original file. That would certainly improve memory consumption but your processing time would greatly be reduced.
Life goes very fast. Tomorrow, today is already yesterday.
|
|
|
|
|
so the method works for not adding duplicates in a file? And how do you remove it the duplicate in the file.
|
|
|
|
|
Yeah, but that is how you will need to do it. You cant just duplicates from a file, unfortunately there is not a function such as File.RemoveDuplicates();
Of course you could read a whole xml file into a datatable and then remove duplicates and re-write it back to the file but you don't want to have to load all the data into memory at once.
So the best way is to create a reader for you xml file. Then read it one entry at the time and concatenate all the values into one string.
Then you either use some sort of collection to store unique entries which you can use to check for duplicates. Or you can write straight to a new file and just check that file for duplicates each time. if a duplicate is found in which ever method you choose then you simply ignore that entry and read the next entry. Then you have either a file will unique entries, or you have a collection of unique entries which you can then write to a new file.
Fairly straight forward concept really.
Life goes very fast. Tomorrow, today is already yesterday.
|
|
|
|
|
that worked out good for me..thank you...
The code:<pre>
XElement duplicate = XElement.Load("wnrom2.xml");
StringBuilder build = new StringBuilder();
Dictionary<string, string> dict = new Dictionary<string, string>();
foreach (XElement dup in duplicate.Descendants("SYNSET"))
{
build.Append(dup.ToString());
}
dict.Add("a", build.ToString());
if (dict.ContainsValue(synsets.Descendants("SYNSET").Last().ToString()) == false)
{
synsets.Save("wnrom.xml");
synsets.Save("wnrom2.xml");
}
</pre>
|
|
|
|
|
Hi,
I have a list view with 5 columns and 4 items in that listView.
When i click on a row ( item) of that listViewi want the whole row to be highlighted blue , and if i select another row, what that it to be selected as well.
So basically i want it to look like the whole row is selected not just the first column item.
here is what i tried :
private void listView1_Click(object sender, EventArgs e)<br />
{<br />
for (int i = 0; i < listView1.Items.Count; i++)<br />
{<br />
if (listView1.Items[i].Selected == true)<br />
{<br />
listView1.Items[i].BackColor = Color.Blue;<br />
}<br />
}<br />
}
BUT this works after I change the selection of the row! for example i select row 1 , and when i select row 2 , then row 1 is highlited . Why guys?
Any suggestions please?
Regards,
Alex
“Be the change you want to see in the world.”
|
|
|
|
|
Learn to edit a post or learn what 'FOAD' stands for.
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
thanks man for the advice. I dont know what "FOAD" means but "FOAD" to you as well for helping me
Regards,
Alex
“Be the change you want to see in the world.”
|
|
|
|
|
al3xutzu00 wrote: for (int i = 0; i < listView1.Items.Count; i++)
{
if (listView1.Items[i].Selected == true)
That's unnecessary, use listView1.Items.SelectedIndex.
Then to apply the color change use listView1.Refresh();.
My failometer is detecting vast quantities of FAIL!
"Its SQL - hardly programming..." (Caslen)
|
|
|
|
|
The SelectedIndexChanged event [^] looks more promising... 
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
This is going on my arrogant assumptions. You may have a superb reason why I'm completely wrong.
-- Iain Clarke
[My articles]
|
|
|
|
|
You frakkin' cheater. You're referring to the documentation!  
It is a crappy thing, but it's life -^ Carlo Pallini
|
|
|
|
|
What? There's documentation! I thought he'd used a deccompiler thingy.
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
williamnw wrote: What? There's documentation!
Yes, but it's usually top secret and can be reached only by elusive things like a Google search. 
It is a crappy thing, but it's life -^ Carlo Pallini
|
|
|
|
|
Wow! Does everyone else know about this?
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
Is there a reason that ListView.FullRowSelect[^] can't be used?
It is a truth universally acknowledged that a zombie in possession of brains must be in want of more brains.
-- Pride and Prejudice and Zombies
|
|
|
|
|
Thanks a lot Dan. This worked perfectly.
Alex 
“Be the change you want to see in the world.”
|
|
|
|
|
Hi,
I have a word file. I want to save the first page as a picture. How can I do it using C# code?
Thanks,
Prajeesh
|
|
|
|
|
Hi
i want to set pk table to GUID in access 2007, how can i do this ?
do must i define column with Text dataType ?
Any idea ?
Thanks
|
|
|
|
|
The UNIQUEIDENTIFIER[^] data type exists for this purpose.
It is a crappy thing, but it's life -^ Carlo Pallini
|
|
|
|
|
Hi Rajesh
yes i know, but in access 2007?
|
|
|
|
|
OK, I got your query wrong in the beginning. You could store it as text and you can use GUIDFromString() and StringFromGUID() functions for manipulation.
It is a crappy thing, but it's life -^ Carlo Pallini
|
|
|
|
|
Hi,
wrong forum. use database forum!
bye
|
|
|
|
|
Hello
I create Enterprise Resource Planning apllication for construction branch.
I need easy to lern, rebuild, flexible and with good performance O/R mapper.
There is many option:
- nHibernate,
- SubSonic,
- XPO,
- Castle ActiveRecord,
- Lightspeed,
- CodeSmith,
- Fluent nHibernate,
- Visual Paradigm ORM.
Probably for begining every orm is good for me but in future when me application grows, not exactly.
Please, share with me yours experiance and let us choose the best solution.
Best regards
Lukas
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
