|
So I'm working on a vendor's database trying to search it. It's not normalized real well and the tables are kind of elephanted up as it was originally an MS Access database that they ported to MSSQL. I cannot modify the database but I can create views. In any case, what I currently have is a recursive string builder assembling a SQL statement that searches across multiple tables and views that I created to try to streamline things (one of my first SQL projects, it's not pretty). As you can imagine, it's sloooow. Can anyone point me towards a tutorial that would help me optimize this search?
Current query and code:
string[] strSearch = TextBox1.Text.Split(' ');
string strSelect = "SELECT INVY.ITEMNUM AS Item, " +
"INVY.DESCRIPTION AS Description, STOCK.QTYONHAND AS Quantity, " +
"INVY.UOM AS 'Issue By', STOCK.LOCATION AS Location, INVY.MODEL AS Model, " +
"INVY.UD2 AS IDGroup " +
"FROM INVY LEFT OUTER JOIN STOCK ON INVY.ITEMNUM = STOCK.ITEMNUM " +
"LEFT OUTER JOIN vw_Koogle_ISSREC_Grouping ON INVY.ITEMNUM = vw_Koogle_ISSREC_Grouping.ITEMNUM " +
"LEFT OUTER JOIN INVVEND ON INVY.ITEMNUM = INVVEND.ITEMNUM " +
"WHERE ";
foreach (string strParam in strSearch)
{
strSelect = strSelect + "AND ((INVY.DESCRIPTION LIKE '%" + strParam + "%') OR " +
"(INVY.NOTES LIKE '%" + strParam + "%') OR " +
"(INVY.MODEL LIKE '%" + strParam + "%') OR " +
"(INVY.UD2 LIKE '%" + strParam + "%') OR " +
"(INVY.UD1 LIKE '%" + strParam + "%') OR " +
"(INVY.ITEMNUM LIKE '%" + strParam + "%') OR " +
"(vw_RRD_Koogle_ISSREC_Grouping.EQNUM LIKE '%" + strParam + "%') OR " +
"(vw_RRD_Koogle_ISSREC_Grouping.NUMCHARGEDTO LIKE '%" + strParam + "%') OR " +
"(INVVEND.VENDORITEMNUM LIKE '%" + strParam + "%')) ";
}
strSelect = strSelect.Replace("WHERE AND", "WHERE") +
"GROUP BY INVY.ITEMNUM, INVY.DESCRIPTION, STOCK.QTYONHAND, " +
"INVY.UOM, STOCK.LOCATION, INVY.MODEL, INVY.UD2 " +
"ORDER BY INVY.DESCRIPTION";
sqlMP2Search.SelectCommand = strSelect;
}
modified 1-Mar-19 11:31am.
|
|
|
|
|
Do you want to get so many tutorial URLs that point to nothing, and might not be able to provide a valid result? I am sure, not.
Please share a sample query with us, that you are running and the result you are getting. It would be really helpful to share the schema with us, and we would be able to help you out with the normalization process, or at least share how you can use other services on SQL Server like stored procedures to minimize the overheads of the queries.
You might also need to incorporate caching services to store the results, instead of having to always rerun the query.
Query Optimization and the SQL Server Cache
Please share more details on this, and we will be able to provide you a good solution—or tutorial, as needed.
The sh*t I complain about
It's like there ain't a cloud in the sky and it's raining out - Eminem
~! Firewall !~
|
|
|
|
|
Updated the question. Hope that helps. Thanks for the feedback!
|
|
|
|
|
With that update in the question, first thing that anybody is going to say is to remove the concatenation and use parameters in the query—it will protect you from SQL Injection.
Secondly, there are so many LIKE operators, why are you using searches on the table, and trying to match every column with the input. A quick tip would be, use a separate search for each column and then try to aggregate the overall results. This would have a little amount of WHERE clause, and the query would end quickly, yours is having multiple OR clauses, which is not letting SQL Server short-circuit the query either and is making it run on each of the records.
I would also not be so sure as to whether any indexing would speed up things, but you can try adding indexes on these columns. Read this for more on that, tsql - SQL Server: Index columns used in like? - Stack Overflow
So, you can start by:
1. Add indexers to the columns as needed. You will know where to add them.
2. Create separate stored procedures to find the query results. Then try to aggregate the result of all queries.
3. Use caching to store the results of the most recent queries.
4. Also, try using full text search capabilities of SQL Server.
Full-Text Search - SQL Server | Microsoft Docs
sql - What is Full Text Search vs LIKE - Stack Overflow
The sh*t I complain about
It's like there ain't a cloud in the sky and it's raining out - Eminem
~! Firewall !~
|
|
|
|
|
It's definitely one of the ugliest things I've ever written but also one of my first. Thanks for the pointers, definitely going to be a big help
|
|
|
|
|
|
|
You're welcome. Should be faster for a search than filtering each table, at the cost of some diskspace.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
HOW CREATE MODULE MANGEMENT OF DOCUMENT IN PHP
|
|
|
|
|
SIMPLE: HIRE A PROGRAMMER AND PAY HIM/HER.
If you have a specific question on PHP, we can help, but we do not provide complete modules.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
First start here[^]
And please don't SHOUT in future - all capitals is considered to be rude on Internet forums don't you know.
|
|
|
|
|
Are there some problems with your keyboard (or just with the Caps-lock)?
|
|
|
|
|
Maybe vision - those little characters are just to hard to see?
|
|
|
|
|
Hmm, interesting explanation...
So, zoom doesn't work either?
|
|
|
|
|
This question is not specific enough to answer well, because the answer is "write the code."
Do you have a more specific question?
Social Media - A platform that makes it easier for the crazies to find each other.
Everyone is born right handed. Only the strongest overcome it.
Fight for left-handed rights and hand equality.
|
|
|
|
|
What else could cause this error beside the sequence not being created?
I have recreated the sequence using SQL code, also recreated it using Oracle SQL Developer "New Sequence" menu but still nothing. I only have a single class and and a single table, being a learning project, I don't want to complicate things. Project made using Maven.
hibernate.cfg.xml:
<?xml version = "1.0" encoding = "utf-8"?>
<!DOCTYPE hibernate-configuration SYSTEM
"http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="hibernate.dialect">
org.hibernate.dialect.Oracle10gDialect
</property>
<property name="hibernate.connection.driver_class">
oracle.jdbc.driver.OracleDriver
</property>
<property name="hibernate.connection.url">
jdbc:oracle:thin:@localhost:1521:xe
</property>
<property name="hibernate.connection.username">
SYSTEM
</property>
<property name="hibernate.connection.password">
testingdatabasepassword
</property>
<mapping resource="com/testing/classes/Person.hbm.xml" />
</session-factory>
</hibernate-configuration>
Person.java
package com.testing.classes;
public class Person {
private int id;
private String firstName;
private String lastName;
private int age;
public Person(String firstName, String lastName, int age) {
this.firstName = firstName;
this.lastName = lastName;
this.age = age;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
}
Person.hbm.xml
<?xml version = "1.0" encoding = "utf-8"?>
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD//EN"
"http://www.hibernate.org/dtd/hibernate-mapping-3.0.dtd">
<hibernate-mapping>
<class name = "com.testing.classes.Person" table = "PERSON">
<id name = "id" type = "int" column = "ID">
<generator class="sequence">
<param name="sequence">CREATE_PERSON_ID</param>
</generator>
</id>
<property name = "firstName" column = "FIRST_NAME" type = "string"/>
<property name = "lastName" column = "LAST_NAME" type = "string"/>
<property name = "age" column = "AGE" type = "int"/>
</class>
</hibernate-mapping>
Sequence:
CREATE SEQUENCE CREATE_PERSON_ID INCREMENT BY 1 START WITH 1 MAXVALUE 9999 MINVALUE 1 CACHE 10;
Testing main:
SessionFactory sessionFactory = new Configuration().configure().buildSessionFactory();
Session session = sessionFactory.openSession();
Person person = new Person("Name", "Last name", 48);
session.save(person);
session.beginTransaction().commit();
modified 18-Feb-19 8:55am.
|
|
|
|
|
I don't use Oracle, so this is just a guess: do you need to grant SELECT permissions on the sequence to the user you're connecting as?
To use a sequence, your schema must contain the sequence or you must have been granted the SELECT object privilege for another user's sequence.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Well, as you can see in my hibernate.cfg.xml, I'm using SYSTEM, so that isn't the problem unfortunately.
Edit:
After some more research I found the problem. All the sites with tutorials google suggested were using either
<generator class="native"> or <param name="sequence" />SEQUENCE_NAME
The problem is that hibernate devs changed <param name="sequence" /> to <param name="sequence_name" />. It is weird that you get sequence does not exist error, instead of something like invalid value for param or something to suggest that is the problem.
modified 18-Feb-19 8:54am.
|
|
|
|
|
To solve this problem you have to change:
<generator class="sequence">
<param name="sequence">SEQUENCE_NAME</param>
</generator>
into
<generator class="sequence">
<param name="sequence_name">SEQUENCE_NAME</param>
</generator>
The value for param was changed from just "sequence" to "sequence_name".
|
|
|
|
|
After you install Oracle 11g XE, it will automatically create some tables (from what I've read, some are used for "Application Express", and you shouldn't drop them), but having them listed in Tables view is really annoying.
Is there a way to filter them out? And why do you use $ in table name for?
modified 16-Feb-19 14:04pm.
|
|
|
|
|
Can't help you with the filtering since I don't work with Oracle anymore, but the V$ is just the naming standard Oracle uses for "Volatile" files, containing statistics and such stuff
|
|
|
|
|
Oh OK, I though they have some special meaning, something that I will have to use in the future too, guess not. Thanks!
|
|
|
|
|
For some reason, last time I searched on google it didn't showed me anything useful (on any of the first 4 pages), but this time, I got the answer I needed.
Solutions:
1. Use a different user then SYSTEM. I didn't though at this, after all I didn't wanted to complicate things on a learning project.
2. Filter each group using %. When I tried it before it didn't worked, but now after I started the DB again, the filter worked as it should have. Maybe there was an error connecting to the DB, don't know, it's good that now it is working.
|
|
|
|
|
When i execute the procedure what i get the output is like -
ALTER PROC [dbo].[PROC_EMPLOYEE_PAY_SLIP]
@EmplopyeeCode varchar(100),
@MonthName varchar(50),
@Year Float
As
Begin
SELECT ISNULL(HR_EmployeeMaster.em_EmplopyeeCode,'') As EmplopyeeCode,
ISNULL(CASE WHEN fa_ComponentMaster.Cm_ADTag = 'A' THEN fa_ComponentMaster.Cm_Name ELSE '' END,'') As Additions,
ISNULL(CASE WHEN fa_ComponentMaster.Cm_ADTag = 'D' THEN fa_ComponentMaster.Cm_Name ELSE '' END,'') As Deductions,
ISNULL(CASE WHEN fa_ComponentMaster.Cm_ADTag = 'A' THEN Fa_MonthSalary.Ms_Amount ELSE 0 END,0) As AdditionsAmount,
ISNULL(CASE WHEN fa_ComponentMaster.Cm_ADTag = 'D' THEN Fa_MonthSalary.Ms_Amount ELSE 0 END,0) As DeductionsAmount
FROM HR_EmployeeMaster
LEFT JOIN Fa_MonthSalary ON Fa_MonthSalary.ms_EmpCode = @EmplopyeeCode
LEFT JOIN Fa_MonthDetails ON Fa_MonthDetails.mo_id = Fa_MonthSalary.Ms_MonCode
LEFT JOIN fa_ComponentMaster ON fa_ComponentMaster.cm_code = Fa_MonthSalary.Ms_CompCode
WHERE HR_EmployeeMaster.em_EmplopyeeCode = @EmplopyeeCode AND
(SELECT CONVERT(CHAR(3), Fa_MonthDetails.mo_name)) = @MonthName AND
Fa_MonthDetails.mon_year = @Year
End
Current Output -
EmployeeCode Additions Deductions AdditionsAmount DeductionsAmount
M1010 ALLOWANCE 960.00 0
M1010 BASIC SALARY 1390.00 0
M1010 OT1 171.35 0
M1010 OTHERS-DED 0.00 1800.00
Required Output -
EmployeeCode Additions Deductions AdditionsAmount DeductionsAmount
M1010 ALLOWANCE OTHERS-DED 960.00 1800.00
M1010 BASIC SALARY 1390.00 0
M1010 OT1 171.35 0
Any help would be appreciated
|
|
|
|
|
|