|
I have a problem where I am creating a data layer for a project, where I have a scenario I don't really know which solution is better. The main function of the project/service is to regularly check for changes in several databases and copy the information to a data warehouse(one big database really).
One scenario is when you add an new table to be tracked in the system. It has a specific task to copy the entire table from the site database to the warehouse. It does this by opening a data reader and then generate a batch INSERT script until it reaches a limit and then it execute the script towards the data warehouse. In pseudo code it looks like this:
using (IDbCommand clientCommand = Connection.CreateCommand())
{
clientCommand.CommandText = clientSql;
using (IDataReader clientReader = clientCommand.ExecuteReader()
{
while(clientReader.Read())
{
AddDataToList(clientReader);
}
if(List.Count >= 100)
{
using (var com = warehouseConnection.CreateCommand())
{
com.CommandText = GetInsertScript(List);
AddedCount += com.ExecuteNonQuery();
}
}
}
}
Its more complicated than that, with more checks and sql code. I see two main issues with this:
1: Database logic in business logic that I want to move over to my data layer. This will make the code testable and easier to understand.
2: I do not like that we wait for data from the client, and then wait for data to be written to the other database, when this could be done in 2 tasks that transfer and receiving information asynchronously with each other via ConcurrentQueue for instance. This speeds up the process, which is also an issue in some cases.
The problem I am having is that I that I cant get all the data in one go and then add it to the warehouse. Sometimes the tables are huge, which can lead to memory issues on the service. So I want to process the data as they come in through the client reader and continuously add them to the warehouse. But, I also want the database logic in the data layer. Adding the data is simple, but retrieving it is a bit more complicated. Researching this issue have given me 2 choices:
Option 1 (Pull from data layer):
I could use the yield keyword to return data as it comes through the reader in my data layer:
public IEnumerator<object[]> GetTableContent()
{
SqlCommand command = new SqlCommand("command", SqlConnection);
using (var reader = command.ExecuteReader())
{
var items = new object[reader.FieldCount];
reader.GetValues(items);
yield return items;
}
}
With this I can use it in a foreach loop, and each time I get a return I add it to the queue. Another task takes from the queue and adds it to the data warehouse. With this I can control the when I want a new result from the site database and not add to many items to the queue if the data warehouse is busy.
Options 2 (Push from data layer):
I can use Actions to push data to the caller:
public void GetTableContent(Action<object[]> dataAction)
{
SqlCommand command = new SqlCommand("command", SqlConnection);
using (var reader = command.ExecuteReader())
{
var items = new object[reader.FieldCount];
reader.GetValues(items);
dataAction.Invoke(items);
}
}
Here I call the method and it will start to give me information through an action instead.
From the 2 options I find the pull method is a little more graceful and it is more inline with how the reader works. But I have also read that people are having issues with this where having yield returns in using have been an issue. That's where option 2 has been suggested. It should be be able to do the exact same thing. So... which option is better. Option 1 or Option 2 or maybe there is another way I haven't even considered?
|
|
|
|
|
To be honest, neither solution really appeals to me. Have you considered using a database server to do the ETL instead? SQL Server, for instance, provides excellent ETL capabilities.
|
|
|
|
|
That is a very interesting suggestion. I cant use that at the moment, because the system is not setup for that. But its clear this is the way to go. Thank you for that!
|
|
|
|
|
I’m looking to start a new project in a field I have not explored before so I’m looking for advice on libraries to use and approaches. E.g. could this be done with Tapi?
Two laptops running Windows (7 or above) both in the same subnet. One is nominated the masted and the other the slave.
The master is to set up a voice link to the other laptop, communications is alway only between these 2 laptops. Must have the ability to mute the mic programmatically. Lastly all communications need to be written to an audio file. There is no requirement to have a video link.
The master also needs to pass data instruction to the slave, this is not connected to the voice communications. The slave needs to acknowledge the instruction from the master.
What libraries would recommend, this is to be a c# project.
Thanks in advance
|
|
|
|
|
I've worked with C#. I want to learn another language besides the C#. I want to choose between Java and Python. Python is capable of doing anything but because of its interpretive nature, it's too slow. On the other hand, Python is the #1 programming language based on benchmarks available online. Is there anything that Python or Java or C# can do in desktop GUI application development while the others can't?
|
|
|
|
|
Python may be too slow, or it may not. It depends on what your application needs to do. An interpretive language works just fine for many problems.
|
|
|
|
|
Broadly speaking the answer is, "no". If you can do it in C# then you can do it in Java or Python. It is just the implementation that is different.
|
|
|
|
|
I hope this is clear. If not, I can explain further.
I have a table that looks like this
I will be running this Linq to SQL query repeatedly
var maxRev = 2;
var jobSequenceSheets = (from jss in dc.JobSequenceSheets
where jss.JobId == jobId &&
jss.RevisionNumber == maxRev
select jss).OrderBy(x => x.Plan)
.ThenBy(x => x.Elevation)
.DistinctBy(c => new { c.Plan, c.Elevation})
.ToList();
So, of you look at the image, given RevisionNumber 2, I should get back only two records with Plan & Elevations of 1A and 2A. There are two rows for 2A because of the Lot. I don't care about the lot.
All I care about for a new feature I'm working on is that I get back a distinct set of Plan/Elevations with the SAME ID'S EACH TIME IT'S RUN. Since there are two rows for 2A, given this query, I should get back only 2 rows (not 3). Can I expect to get back the same ID's each time?
Does DistinctBy use the FIRST matching row it finds? I would expect this query to give me back rows 22607 and 22608 each run.
Thanks
If it's not broken, fix it until it is.
Everything makes sense in someone's mind.
Ya can't fix stupid.
|
|
|
|
|
Since you are creating a new instance for each item you check in the comparer, it will return every instance as a new, distinct element, since it will compare instance references and by definition new returns different references each time it is called.
Have a look here: C# – DistinctBy extension[^] - it explains the extension method quite well.
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Not quite. An anonymous type uses value equality, not reference equality:
Because the Equals and GetHashCode methods on anonymous types are defined in terms of the Equals and GetHashCode methods of the properties, two instances of the same anonymous type are equal only if all their properties are equal.
Also, the DistinctBy operator which was added in .NET 6 uses a different approach from the blog you linked to:
runtime/Distinct.cs at ebba1d4acb7abea5ba15e1f7f69d1d1311465d16 · dotnet/runtime · GitHub[^]
And the answer will also depend on whether the DistinctBy method gets translated to SQL, or evaluated on the client.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
It depends.
Are you using the DistinctBy method added in .NET 6, or a different implementation?
And does the ORM you're using translated DistinctBy to SQL, or does it evaluate it on the client?
If it evaluates it on the client, and you're using the .NET 6 method or something equivalent, then technically it will return the first item it encounters within each group.
However, this is an implementation detail, and you cannot rely on it. And since you don't order by the ID, you can't guarantee that the database will return the records in any ID-related order.
If you always want the lowest ID, then you need to be explicit:
var jobSequenceSheets = dc.JobSequenceSheets
.Where(jss => jss.JobId == jobId)
.Where(jss => jss.RevisionNumber == maxRev)
.GroupBy(jss => new { jss.Plan, jss.Elevation }, (_, items) => items.OrderBy(jss => jss.ID).First())
.OrderBy(jss => jss.Plan).ThenBy(jss => jss.Elevation)
.ToList();
NB: Depending on your ORM, you might need to stick an .AsEnumerable() between the second .Where(...) and the .GroupBy(...) to force client evaluation of the grouping.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Hello everyone, I want it to read the contents of the txt file I selected from the list box and then draw a picture in the picturebox according to the values it gets from the txt file. can you please help?
|
|
|
|
|
Help with what? You haven't asked a question; you've just given us a vague requirement.
You need to start by deciding what syntax your text file is going to use.
You then need to write the code to parse the file and convert the contents to some form of actionable information.
Finally, you will need some code to take the parsed information and turn it into a drawing.
So far, you don't appear to have done any of these steps.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
C# windows will list the txt files in the file path I gave in the listbox I created on the form.
When I select one of the txt files listed, it will read the coordinates in the content of this file line by line and draw in the picturebox.
|
|
|
|
|
I see. So you've totally ignored the advice I gave you, and have still made precisely no effort. 
Whilst we're happy to help you fix the code you've written, nobody here is going to do all the work for you.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Use turtle graphics. Read the equivalent (turtle) "command codes" (that you create) from a text file.
GitHub - nakov/TurtleGraphics.NET: C# Turtle Graphics library - for teaching kids to code
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
|
|
|
|
|
And?
What have you tried?
Where are you stuck?
What help do you need?
Just posting the end result without telling us where you are at the moment doesn't help anyone - we have no idea what existing design we have to fit into!
If you are having problems getting started at all, then this may help: How to Write Code to Solve a Problem, A Beginner's Guide[^]
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
2yvdecv.jpg[^]
I designed the form you see in the picture and I list the files in the file path I specified in the Listbox. It needs to read the content of the file with Streamreader and draw a picture in the picturebox accordingly. I'm reading the file but I can't draw the picture.
DirectoryInfo dir = new DirectoryInfo(@"C:\\Users\\BS\Desktop\\Yeni_klasör");
DirectoryInfo[] dirs = dir.GetDirectories();
FileInfo[] files = dir.GetFiles();
foreach (FileInfo file in files)
{
listBox1.Items.AddRange(files);
}
StreamReader str = new StreamReader(listBox1.SelectedItem.ToString());
string satir;
List<string> veriler2 = new List<string>();
while ((satir = str.ReadLine()) != null)
{
veriler2.Add(satir);
}
I have a code block like this. I want it to read the numbers in the txt file and draw a picture in millimeters accordingly. Can you help me please?
|
|
|
|
|
And you STILL haven't explained what syntax the text file contains, nor what you have tried, nor where you are stuck.
Nobody can help you if you don't give us the information we need to help you.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
I forgot I'm sorry. I am converting dxf file to txt format. The content of this txt file is as follows:
Is an example
Line
0
0
100
0
False
0
Line
100
0
100
100
False
0
Line
100
100
0
100
False
0
Line
0
100
0
200
False
0
How can I draw a picture according to the value it receives after reading this file?
|
|
|
|
|
Member 15570952 wrote: How can I draw a picture according to the value it receives after reading this file?
It all depends on the meaning of these values!
|
|
|
|
|
Forget the drawing for now.
You have to create a parser for the file. That parser is going to create an object "graph", which is the list of commands your drawing code is going to interpret to tell it what/how to draw.
You have to ask yourself what does each line in the file mean and convert those items to data structures. For example, the parser is going to com across a line with "Line" in it. To create a Line structure, you need to know the X and Y coordinates of both endpoints of the line, so you need four values. The parser should expect those four values on the lines after "Line". Read those values and place them into the structure. Once the values are filled in, add the structure to a List that holds those structures. This list is what the drawing code is going to need to tell it what to do.
What odes the "False" and the "0" after it mean? Your parser is going to have to deal with those too, as appropriate.
Why do you need to do this? Because of the way Windows works. Windows calls your app to tell it to draw itself, and your window needs the data to redraw itself over and over again. For example, if another window is dragged over the top of your window, Windows will tell your app to redraw itself repeatedly, as the other window is being dragged over the top of yours.
|
|
|
|
|
Message Closed
modified 25-Jun-22 13:13pm.
|
|
|
|
|
Not getting it ... What was your post? In some other thread?
|
|
|
|
|
Hello there , I want to create a simple paint application (photo editor tool) in winforms C# which should draw basic shapes (Rectangle, Ellipse, Line and Arrow only). Also, it should draw a transparent text box to write text and all these shapes including the text should be draggable or movable i.e. we should be able to select a shape or text and move it to other location on the screen. Further more it should have undo, redo, save and save as options also.
In simple words, it should have:
(rectangle, ellipse, line , arrow, text , undo, redo, save, save as, movement of shapes)
Using : C#- Winforms
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
