How to check duplicates in string array c#

Question

1.00/5 (2 votes)

See more:

C#

How to check duplicates in string array c#..Also show a message if duplicate present....

Posted 14-Apr-13 21:37pm

Korathu 2

Updated 18-Nov-20 6:15am

Add a Solution

6 solutions

Solution 2

There are several options. One method is:

First sort the array.
Then, foreach item: if it is equal to the next one then you have a duplicate (and you may show the message).

Posted 14-Apr-13 21:50pm

CPallini

Solution 3

You can use Distinct() as Shmuel mentioned.
If you want to display the duplicated items , then you can use Linq

C#

 string[] array = { "First", "Second", "Third", "First", "Third" }; 
var i= CheckforDuplicates(array);

C#

public bool CheckforDuplicates(string[] array)
      {
          var duplicates = array
           .GroupBy(p => p)
           .Where(g => g.Count() > 1)
           .Select(g => g.Key);


          return (duplicates.Count() > 0);



      }

where the variable duplicates contains the list of repeated items

Posted 14-Apr-13 21:54pm

Naz_Firdouse

Comments

Korathu 2 15-Apr-13 4:27am

None worked ...

Naz_Firdouse 15-Apr-13 5:13am

wt happened?
if the variable is true,it means there is a duplicate value...
what is the issue here?

Solution 4

Similar to Naz's Solution:

C#

private static int CountDuplicates(string[] array)
{
  HashSet<string> stringSet = new HashSet<string>();
  int numDups = 0;
  foreach (var item in array)
  {
    if (stringSet.Contains(item))
    {
      ++numDups;
    }
    else
    {
      stringSet.Add(item);
    }
  }
  return numDups;
}

// this returns an IEnumerable with both the strings and the repeat counts (might be useful for reporting...)
private static IEnumerable<KeyValuePair<string, int>> FindDuplicates(string[] array)
{
  Dictionary<string, int> stringSet = new Dictionary<string, int>();
  foreach (var item in array)
  {
    int count;
    if (stringSet.TryGetValue(item, out count))
    {
      stringSet[item] = count + 1;
    }
    else
    {
      stringSet[item] = 1;
    }
  }
  return stringSet.Where(p => p.Value > 1);
}

The execution time of this is O(n) in the length of array.

Posted 15-Apr-13 9:16am

Matt T Heffron

Comments

aarnone2 3-Mar-18 9:15am

Actually this is order (N log N)

The stringSet.Contains operation is of log N complexity

Matt T Heffron 3-Mar-18 12:52pm

Nope. HashSet<t>.Contains() is O(1).
https://msdn.microsoft.com/en-us/library/bb356440(v=vs.110).aspx

Solution 5

        static bool hasDuplicates(string s)
        {
            int[] countOfThisCharacter = new int[256];
            bool b = false;
            bool nonunicode = false;

            for (int i=0; i<s.Length; i++)
            {
                if (((int)(s[i]))>255) { nonunicode = true;  break; }
                if ((countOfThisCharacter[s[i]])>0) { b = true; break; }
                countOfThisCharacter[((int)(s[i]))]++;
            }
            if (nonunicode==true)
            {
                b = false;
                Dictionary<string, string> lst = new Dictionary<string, string>();
                for (int i = 0; i < s.Length; i++)
                {
                    if (lst.ContainsKey(s[i].ToString())) { b = true; break; }
                    lst.Add(s[i].ToString(), s[i].ToString());
                }

            }

            return b;
        }

// this is order (N) complexity for strings without Unicode characters
// and order (N log N) complexity for strings that have Unicode characters
//

Posted 3-Mar-18 3:13am

aarnone2

Updated 3-Mar-18 8:55am

v2

Comments

Dave Kreskowiak 3-Mar-18 9:53am

Asked and ANSWERED OVER FOUR YEARS AGO.

aarnone2 3-Mar-18 10:15am

Yeah but this answer is better. It's the only O(N) way to do this.

Dave Kreskowiak 3-Mar-18 12:48pm

Yeah, two problems.

First, you're checking for duplicated characters within a string, NOT if there are duplicate strings within an array of strings. I'll grant that the original question leaves a bit of room for interpretation as to which situation he/she was talking about.

Second, answering a really old question pops it back up to the top of the QA queue. That's something we don't like because it results in more answers to really old questions, keeping it popping up to the top of the queue.

Matt T Heffron 3-Mar-18 13:19pm

Since the OP says "string array", there doesn't seem to be any ambiguity about the problem statement. So, as you said, this solution isn't for this problem.

Dave Kreskowiak 3-Mar-18 13:44pm

Considering the answers given cover both cases, there DOES seem to be an ambiguity. Most answers cover the string[] case though, not the string.chars[] case.

#realJSOP 3-Mar-18 14:27pm

Not if you look at the OP's reply to one of the solutions.

Dave Kreskowiak 3-Mar-18 15:52pm

The only reply I saw was the one that said "None worked ...". That still left the question vague.

Asking questions is a skill, one the OP apparently didn't have at the time.

aarnone2 3-Mar-18 15:04pm

1. Not that it matters, but my approach is considerably faster for the string.chars[] case. I ran tests, and it's over 5 times faster.

2. I learned what a HashSet is, and can use that rather than a dictionary in the case where I need to use a list.

number of iterations is: 10000

10.8274
10.0563

number of iterations is: 100000

17.0797
97.2567

number of iterations is: 1000000

174.4826
959.5027

number of iterations is: 10000000

1652.4296
9510.2869

Press any key to continue . . .

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ConsoleApplication1
{
class Program
{

static bool hasDuplicates(string s)
{
int[] countOfThisCharacter = new int[256];
bool b = false;
bool nonunicode = false;

for (int i=0; i<s.Length; i++)
{
if (((int)(s[i]))>255) { nonunicode = true; break; }
if ((countOfThisCharacter[s[i]]) >0) { b = true; break; }
countOfThisCharacter[s[i]]++;
}
if (nonunicode==true)
{
b = false;
HashSet<string> lst = new HashSet<string>();
for (int i = 0; i < s.Length; i++)
{
if (lst.Contains(s[i].ToString())) { b = true; break; }
lst.Add(s[i].ToString());
}

}

return b;
}

private static bool HasDuplicates(string s)
{

HashSet<string> stringSet = new HashSet<string>();
bool hasDups = false;
for (int i=0; i<s.Length; i++)
{
if (stringSet.Contains(s[i].ToString()))
{
hasDups = true;
break;
}
else
{
stringSet.Add(s[i].ToString());
}
}
return hasDups;
}

static void runTest(int i2)
{
Console.WriteLine("number of iterations is: " + i2.ToString());
Console.WriteLine("");
Console.WriteLine("");

bool b;
b = false;
DateTime DateTime1 = System.DateTime.Now;
for (int i = 0; i < i2; i++)
{
b = hasDuplicates("The quick brown fox jumped over the lazy dogs.");
}
DateTime DateTime2 = System.DateTime.Now;
Console.WriteLine((DateTime2 - DateTime1).TotalMilliseconds);
b = false;

DateTime DateTime3 = System.DateTime.Now;
for (int i = 0; i < i2; i++)
{
b = HasDuplicates("The quick brown fox jumped over the lazy dogs.");
}
DateTime DateTime4 = System.DateTime.Now;
Console.WriteLine((DateTime4 - DateTime3).TotalMilliseconds);

Console.WriteLine("");
Console.WriteLine("");

}

static void Main(string[] args)
{

runTest(10000);
runTest(100000);
runTest(1000000);
runTest(10000000);

}
}
}

Solution 7

While using linq is the best answer, here's an alternative:

C#

string text = "1,2,3,4,4,5";
string[] parts = text.Split(',');
HashSet<string> hset = new Hashset<string>();
foreach(string item in parts)
{
    hset.Add(item);
}
if (hset.Count < parts.Length)
{
   // there's a duplicate string
}

A HashSet collection does not allow duplicates, but at the same time, it will not throw an exception when you try to add a duplicate.

If you want to know WHICH string(s) is/are a duplicate, you could change the code to be something like this:

C#

string text = "1,2,3,4,4,5";
string[] parts = text.Split(',');
HashSet<string> hset = new Hashset<string>();
List<string> duplicates = new List<string>
foreach(string item in parts)
{
    int oldCount = hset.Count;
    hset.Add(item);
    if (hset.Count == oldCount)
    {
        if (!duplicates.Contains(item))
        {
            duplicates.Add(item);
        }
    }
}
if (duplicates.Count > 0)
{
   // list the duplicates 
}

Keep in mind that the code above is case sensitive, so "Text" and "text" will be seen as different values. If that matters, and it if it won't have side effects elsewhere in your code, you can always call ToUpper or ToLower on the original text varaiable before processing it for duplicates.

Lastly, punctuation in a sentence can obfuscate duplicate words, so you'd probably want to remove punctuation characters from the original string, or remove them when adding them to either the hset and the duplicates collections (in the example above).

Posted 3-Mar-18 5:36am

#realJSOP

Updated 3-Mar-18 8:33am

v4

Comments

Richard MacCutchan 3-Mar-18 11:44am

Uh oh John, check the date of the question.

#realJSOP 3-Mar-18 11:48am

I don't care. :)

Richard MacCutchan 3-Mar-18 11:48am

Rebel.

#realJSOP 3-Mar-18 11:53am

To show that I'm willing to work within the system, I posted a suggs/bugs message:

https://www.codeproject.com/Messages/5496460/Q-A-Suggestion.aspx

:)

#realJSOP 3-Mar-18 11:56am

I think I'm gonna go back to the first question ever posted in Q/A and answer it.

aarnone2 3-Mar-18 12:19pm

That solution is O(N log N). Contains is a log N operation.

The time required to create the hashtable and add entries adds complexity. Searching hashtables is close to O(1) in most cases, but isn't exactly O(1)

The hash we're looking at in the most common case is very small--it can't be more than 255 entries for a non-unicode string.

So I think my approach of using an array of 255 integers for all non-unicode strings is better.

#realJSOP 3-Mar-18 14:26pm

Your solution doesn't even answer the OP's question. Based on his replies to other solutions, he wanted to find duplicate words in a text string. With that in mind, your solution won't even work.

#realJSOP 4-Mar-18 7:47am

So, did you 1-vote my answer, or did the same guy that 1-voted your answer do mine too? That's just f*ckin childish.

aarnone2 4-Mar-18 9:27am

Wasn't me. I was the guy who so childish as to remove his website from the source just to prevent this discussion from hurting his online reputation.

Add a Solution

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Shmuel Zang · Accepted Answer · 2013-04-14T21:47:00

Solution 1

You can use the Distinct method. Something like:

C#

string[] myStrings = // ...

if (myStrings.Distinct().Count() != myStrings.Count())
{
    // Show message
}

Posted 14-Apr-13 21:47pm

Shmuel Zang

Comments

Ian A Davidson 15-Apr-13 4:03am

Good answer. +5.

Himanshu Yadav 20-Sep-13 22:08pm

But how to convert in array if string is like "we are not a employee but we come in employee table" without using split('') method.......please help me out

Shmuel Zang 15-Apr-13 4:27am

Thank you Ian.

Shmuel Zang 22-Sep-13 15:43pm

Try this: http://stackoverflow.com/questions/9067152/splitting-string-without-string-split

farrukh azad 3-Sep-22 8:35am

Nice , It work for me

How to check duplicates in string array c#

6 solutions

Solution 1

Solution 2

Solution 3

Solution 4

Solution 5

Solution 7

Add your solution here

Preview 0

How to check duplicates in string array c#

6 solutions

Solution 1

Solution 2

Solution 3

Solution 4

Solution 5

Solution 7

Add your solution here

Preview 0

Existing Members

...or Join us