What's the best way to scrape a web page that has AJAX/dynamic loading of data?
For example: scraping a webpage that presents 20 images on load, but when a user scroll down the page it loads more images (sort of like Facebook). In such a case how do you scrape all the images, not just the first 20?
Now I process the data synchronized, when recieving the data it deal data.
But I found the data is high frequently recieved, And it waste too much time to process data.
So, I hope to process data asynchronous and ensure the data sequence, not to block the trecieve thread.
I read some arcitles, I get two methods:
1) when recieving data ,right now use " task.factory.startnew..."
2) when recieving data , "enqueue" the data to ConcurrentQueue, According to different data type, i use 8 queues. And the the program is started,
starting 8 threads to dequeue data for every ConcurrentQueue and process it.
I face a problem when run my windows application at the first time( it is take a long time to load - up to 7 seconds) after that when i close it an open again it may take 2-3 seconds, I do some search and find the ngen.exe is the best solution.
Please let me know if the Ngen.exe the best solution or not, if yes how to use it in VS2010, if no what is the best solution.
The main form is the login form which contains the following controls:
1. 3 text box for login info as the following:
*. User Name text-box.
*. Domain text-box. (UserName= userLogin@Domain)
*. Password text-box.
2. Login button.
3. User Name Label.
4. Password Label.
5. Version Label.
And this form is take load time when open it in the first time.
Please Let me know how to use the NGEN.exe to solve this issue.
This is normally done in your application installer. After your application is installed, you run NGEN on it to get the binaries built. This is normally an automatic option in modern installation builder tools like InstallShield and InnoSetup.
You're also going to improve performance by evaluating what your application is doing on startup, not just counting controls. What is going on in your Program.cs? What is going on in your Form.Load event? How big is your resulting executable when you compile it? Are you doing any database work? Are you using Entity Framework? There is a "warm-up" time when EF first starts.
ngen does help, I´ve used it in the past. However, make sure that your loadup isn´t taking unnecessarely long.
NGen is not a miracle thing and probably is more a workaround than a real solution.
Not sure if it exists still, but in winforms you could have a look at SuspendLayout, ResumeLayout.
Also, in our case a while back, the slow loading was due to thousands and thousands of records that had to be loaded in linked lists. When I convinced the client to change in a small way, the way they were working we could load each list on runtime when (and indeed if) the user needed it. The startup time changed from a few seconds to instantly.
Last thing is that you have to make sure the hardware is sufficient for that version of the .Net framework (and what the application is used for)
Hi, this is sadish6 I want the algorithm for Multi-label Classifiers based Associative Classification (MCAC) in C# or VB.NET can any one please help me by providing the code
this link can be useful (https://www.dora.dmu.ac.uk/bitstream/handle/2086/10120/Neda-Thesis-FinalVersion.pdf?sequence=1)
I currently worke on Programm working with the DevComponents.DotNetBar.DockingContainers, those Containers have a lot of functionality. My Problem is now, since i'm working with them as dynamic objects, e.g. a new usercontrol Needs to be shown -> Check if a DockingContainer is empty -> if not create one and put the uc in the new Container, i get several issues that i'm not able to solve.
1. The DockingContainers get deleted from time to time, e.g. uc is not needed anymore -> dispose it and delete the Container.
In this case my dockSite gets shrunk by the amount the Container was.
I have a "Workaround" for that but ist not satisfying as the other Containers are flashing while i resize.
Code for delete given:
/// Deletes the container.
/// <paramname="sender">The sender.</param>
/// <paramname="e">The <seecref="EventArgs"/> instance containing the event data.</param>
/// <exceptioncref="System.NotImplementedException">Something went terribly wrong</exception>
privatevoid DeleteContainer(object sender, EventArgs e)
DockContainerItem _tmp = (DockContainerItem)_currentClickedBar.Items;
PanelDockContainer _tmpPanel = (PanelDockContainer)_tmp.Control;
if (_tmpPanel.Controls.Count != 0)
... // Codeblock that is unfinished and not relevant
//Wenn der Container leer ist kann er einfach gelöscht werden
_dockingPanels.Remove(_dockingPanels.Find(dp => dp.Name == _tmpPanel.Name));
_dockingContainers.Remove(_dockingContainers.Find(dc => dc.Name == _tmp.Name));
_barContainer.Remove(_barContainer.Find(bc => bc.Name == _currentClickedBar.Name));
DockSite _ds = (DockSite)_currentClickedBar.DockedSite;
int _width = _currentClickedBar.Size.Width;
int _height = _currentClickedBar.Size.Height;
//delete the Container
//resize the DockSite
_ds.Size = new System.Drawing.Size(_ds.Size.Width + _width, _ds.Size.Height + _height);
_currentClickedBar = null;
2. Another Problem i have is that if i undock a Container, dock it into another one everything gets pretty messed up as i can't undock it out of the Container again and the old one stays inoperable for some reason.
It wasn't inoperable, i just messed up the Text property of the DockingContainerItem, therefore i couldn't see it in the "Tabs"
Here the other code for my Containers:
/// Creates the named docking container.
/// <paramname="NameContainer">The name container.</param>
/// <paramname="NameBar">The name bar.</param>
/// <paramname="NamePanel">The name panel.</param>
publicvoid CreateNamedDockingContainer(string NameContainer, string NameBar, string NamePanel)
// Create the new Bar and set the specific Settings
Bar newBar = new Bar("neuer Container");
newBar.Name = NameBar;
newBar.LayoutType = eLayoutType.DockContainer;
newBar.GrabHandleStyle = eGrabHandleStyle.Caption;
newBar.Stretch = true;
newBar.CanCustomize = false;
newBar.CanAutoHide = false;
newBar.MouseClick += bar_Click;
newBar.BarUndock += barContainer_BarUndock;
//Create the dockContainerItem
DockContainerItem tempDock = new DockContainerItem();
tempDock.Name = NameContainer;
//Create the dockingPanel
PanelDockContainer tempPanel = new PanelDockContainer();
tempPanel.Name = NamePanel;
tempPanel.MouseClick += panelDockContainer_Click;
//Create a new Control for the ContainerItem and reset it to the panel
tempDock.Control = new Control();
tempDock.Control = tempPanel;
//add the dockContainer to the bar, the bar to the dotNetBarManager and set the dockposition
Any help or experience with those DockingContainers would be appretiated
C# / VS2013
if(this.signature != "")
MessageBox.Show("This is my signature: " + Environment.NewLine + signature);
MessageBox.Show("404-Signature not found");
After you assign the DataSource to the Report, I want to assign to the label, textbox, xrTableCell1,... Report should I do? or I want to DataBinding, currently running it does not appear at all records
// After successful connection Northwind// Create an empty report.
XtraReport1 report = new XtraReport1(); string sql = "select * from Customers";
DataSet ds = new DataSet();
SqlDataAdapter da = new SqlDataAdapter(sql, cnn);
report.DataSource = ds;
report.DataAdapter = da;
report.DataMember = Customers; // Table Customers of Northwind