Click here to Skip to main content
15,899,314 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hai,

I have developed a web page to get all comments from twitter. Using below code i have achieved successfully. Here problem is, This code fetch few comments only. but when we search in twitter means it shows more than a lakhs comments. in twitter automatically updates the comments when we scroll the page. but in c# we can not achieve it. i need to fetch 10000 comments at least.

Can you help me anyone???

C#
protected void Button1_Click(object sender, EventArgs e)
        {
            if (!string.IsNullOrEmpty(TextWeburl.Text))
            {
                try
                {
                    string url = string.Empty;
                    url = "https://twitter.com/search?q=" + TextWeburl.Text + "&src=typd";

                    var getHtmlDoc = new HtmlWeb();
                    var document = getHtmlDoc.Load(url);
                    var tags = document.DocumentNode.SelectNodes("//p");
                    if (tags != null)
                    {
                        foreach (var sitetag in tags)
                        {
                            //txtFull.Text += sitetag.InnerText + "\n";
                            if (sitetag.Attributes.Count != 0)
                            {
                                if (sitetag.Attributes["Class"].Value == "js-tweet-text tweet-text")
                                    txtFull.Text += sitetag.InnerText + "\n" + "\n";
                            }
                        }
                    }

                }
                catch (FileNotFoundException ex)
                {
                    throw ex;
                }
                catch (SyntaxErrorException ex)
                {
                    throw ex;
                }
            }
        }
Posted

1 solution

I have not worked with Twitter APIs, but I am certain that it loads comments from the server when user scrolls. AFAIK, you can not get those commenting via scraping like methods. You probably need to send requests to twitter or look into their APIs for a specific method that can help.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900