|
Whichever engine you are using, if you are doing fairly complicated regex expressions…
Create a lot of positive and negative test cases for your uses. That way if you ever port it or upgrade the regex engine, you will know it is still working.
I upgraded one toolset on Windows that totally changed how it handled \r\n line termination between versions. I wrote all of my test cases once I realized the mess that update created.
|
|
|
|
|
thanks for great tips![Rose | [Rose]](https://codeproject.global.ssl.fastly.net/script/Forums/Images/rose.gif)
diligent hands rule....
|
|
|
|
|
I am kicking off a new application that will allow the local SQA and Engineering departments to test a new product. Part of this is to ponder some new ideas - add a database to save test results, etc. I'm also considering embedding python into the application such that the tests would be driven by script rather than some embedded file format. But, this is a new area for me, so can anyonw point out real life examples where this ability was useful?
Charlie Gilley
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
Has never been more appropriate.
|
|
|
|
|
I am kicking off a new application that will allow the local SQA and Engineering departments to test a new product. Part of this is to ponder some new ideas - add a database to save test results, etc. I'm also considering embedding python into the application such that the tests would be driven by script rather than some embedded file format. But, this is a new area for me, so can anyonw point out real life examples where this ability was useful?
Charlie Gilley
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
Has never been more appropriate.
|
|
|
|
|
charlieg wrote: But, this is a new area for me, so can anyonw point out real life examples where this ability was useful?
Forget Python and use Lua . Lua was 100% created for this and you don't have to worry about security issues as much out of the box.
Jeremy Falcon
|
|
|
|
|
I'll look into it. Thanks.
Charlie Gilley
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
Has never been more appropriate.
|
|
|
|
|
Oh, to actually answer your question that I quoted as far as usefulness. Having macros in an application can be very useful, just depends on the app and use case. Photoshop has actions, which is the same thing and some photographers couldn't live w/o it. Granted, it's not code based but most hardcore Photoshop users aren't coders.
As far as the usefulness of having testing type scripts/macros, guess it would depend on the end user and the use case of what's being automated. If you're using Lua, for instance, you can 100% tie that into controlling the UI of your app, just like Photoshop does with their actions.
If the end users are engineers, it could be very useful. I do web development these days, but having test and code coverage is something that no dev should forget for an enterprise application.
So, long winded way of saying, just depends. 
Jeremy Falcon
|
|
|
|
|
having used lua, I can concur that:
- it's fast
- it's small
- it's powerful
but I can also say that it's esoteric and likely "not like other programming languages your org is using", especially if you were considering python.
it's built-for-purpose, and good at that purpose, but I hated writing code for the Lua runtime. Always felt kludgy, and I was never sure exactly what I was doing wrong when I was doing something wrong. Perhaps the latter would have faded with time, but with quite a few programming languages under my belt (C, C++, Tcl, Python, PHP, C#, F#, JS; plus the ones people will discount like SQL & sh, and even the ones I've used less like Ruby, Perl), Lua was (imo) unintuitive and painful to use. Could well be a personal problem as I know other people who love it - but if you get this kind of feedback from the people who would be writing code for the platform, I'm here to nod and agree.
------------------------------------------------
If you say that getting the money
is the most important thing
You will spend your life
completely wasting your time
You will be doing things
you don't like doing
In order to go on living
That is, to go on doing things
you don't like doing
Which is stupid.
|
|
|
|
|
Totally agree that Lua isn't my favorite language at all. But, if the goal is to integrate within an application then using a language specifically designed for that is a lot better and safer than one that's not - which is the point. There's no telling what package you can get from the python world, etc.
It's meant to automate, no different than VBA.
Jeremy Falcon
|
|
|
|
|
I hear you on the automation part.
My experience has been that the host has full control over the sanbox in all 3 embedded languages we've used (py, js, c#). If we're concerned about embedding and security, the place for most scrutiny is the code users are putting in 😉
------------------------------------------------
If you say that getting the money
is the most important thing
You will spend your life
completely wasting your time
You will be doing things
you don't like doing
In order to go on living
That is, to go on doing things
you don't like doing
Which is stupid.
|
|
|
|
|
charlieg wrote: embedding python into the application such that the tests would be driven by script rather than some embedded file format
There are multiple QA testing frameworks. I am not referring to developer unit testing frameworks but rather applications that can interact with applications themselves. That includes UI and Rest at least since I have seen both of those. Probably others.
So those are already in use but are not sufficient?
|
|
|
|
|
Well, I'm not actually testing the UI. What I am doing is developing a test system (hardware) that interfaces to the unit under test. I have heard that "it's useful to embed a scripting language" into an application, allowing users to develop their own procedures and what not.
I'm having trouble getting my head around the general concept. More to follow below.
Charlie Gilley
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
Has never been more appropriate.
|
|
|
|
|
We use TestComplete | SmartBear Software to test a legacy application whose source code just acquired. It is a Delphi 7 application, so we use the deep integration through the executable compiled with debug info. Either it makes a macro recording like script, or one can write a script using one of the supported languages
|
|
|
|
|
I think Python is the best idea for embedding scripting.
Fusion 360 supports add-ins and scripts written in Python and they are dropping support for other languages.
I use an addin for OneNote that lets me write scripts in Python.
I have heard of quite a few other projects that also support Python.
Embedding a python interpreter is relatively easy.
An aside, referring to another comment about Lua:
At a company I worked for they had use Lua for several things. They constantly regretted that choice and were trying to untangle the mess when I left.
|
|
|
|
|
I don't think I made my question very clear. I'm not testing a UI or program, I am testing a physical system. Due to the complexity of the system, a system is needed to generate input, record output and compare to the expected results. Let's say I pick python. Embedding python into the application is straightforward. Then what?
Based on my reading, it appears I have to develop an API for the real application that exposes functionality/services to python via an extension (written in C++ most likely). Am I groking this correctly?
"I have heard of quite a few other projects that also support Python" - Julie, any specific projects? I'll continue my search.
Charlie Gilley
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
Has never been more appropriate.
|
|
|
|
|
Then you have no choice but to expose an interface of sorts. If it's an API, first thing you need to decide of what type of bindings are you going to use.
If it were me, I'd toss a tiny web server on the machine and knock out some rest APIs. Even routers have a webserver on them, and they don't have much processing power. So, it's not like it'll take beefy hardware for one or two automating users. You can find a stripped down tiny one or just cook one up in Node (my favorite, it's faster) or Python on the machine itself.
But this way, your interface/gateway would be language agnostic. Just as long as it can send and receive web requests, which is most languages.
Jeremy Falcon
modified 10-Oct-23 13:17pm.
|
|
|
|
|
Oh this is an excellent idea, embedding a web server. The system we're building already has more than enough capacity to handle something like that.
Charlie Gilley
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
Has never been more appropriate.
|
|
|
|
|
charlieg wrote: I am testing a physical system
ah...ok.
charlieg wrote: Based on my reading, it appears I have to develop an API for the real application that exposes functionality/services to python via an extension
The specifics do in fact depend on the target.
But yes other than that the idea would be correct.
Although I would look more closely into who will be writing those tests also.
I created a system that allowed QA testers to write Word docs which contained individual Test Cases. Between me and the QA testers we came up with a specific form that the Test Cases would follow.
The system tested a phone service box (can't remember the general name but back then companies would buy one of these and stick it in a closet to provide phone/internet.)
The system consisted of the following
1. Various helper libraries/tools. In particular to run two different SSH tools (two were needed because two instances of the same would not work correctly at the same time.) Those were used both to test and drive the hardware.
2. A third party tool called AutoIt. Which still exists. It was used to drive the the UI.
3. The parser/interpreter for the Word docs. Written in perl.
The parser would emit code for AutoIt. I think maybe AutoIt drove everything.
Long time ago.
Only interesting bit is that the hardware did not support IP directly. The UI talked via IP but via a tunneling protocol on one of the other channels. The IP rate was something like 40 bytes a second. (Not meg, not k.))
Worked well because of the two testers one could not program at all. The other could get by but from the QA role using the Word as the source made it much easier to document what was being accomplished for the customer.
-----------------------------------------------
I have done quite a few other solutions with dynamic user driven code. But not for hardware.
I have done it both in C# and Java. For the second the dynamic code was Java. For the first the code was both C# and VB.
At least for Java and C# both support a native API interface.
I worked on one system in C++ which used a knowledge base solution (when those were big) which had its own language. The knowledge base would produce C++. Then I wrote APIs for the difficult pieces which the knowledge base rules would call. One problem with that system is that it did not account for the size of the rule base that we were using (or perhaps the person doing that didn't read all of the docs.) It ended up producing C++ methods that were too big for the compiler to handle. (I had to hack a solution to refactor those those before handing off to the compiler.)
--------------------------------------------------
The problems with such systems.
1. How are programming errors handled? What and how will the user see such problems?
2. How are execution errors handled? When it runs it fails so then what?
3. The people writing the tests will need to some non-trivial programming skills. And this impacts the next part.
People do not tend to excel even if it is possible for them to do so. They just want to get it done. Nothing wrong with that. And for the most part it is better since trying to achieve perfection is seldom worth it.
That however means that, based on the programming skills, they will do things that your system might not be expecting. And that can cause problems.
For example spinning up threads. Or making http/rest/tcp calls to external systems but not doing error handling correctly. Or even silly things like doing a set look up by using a for loop but doing in thousands of times in one execution.
Then they will want you to explain why it isn't working. Or why it is slow.
That is a problem if you did not expect that and plan for it. So you need to plan and I would suggest explicitly document the following
1. Limitations of the system
2. Who is responsible for failures.
3. Escalation process.
As for those external APIs that you might need I would suggest that you also collect plenty of stats:
1. The time it took to execute.
2. When it was called
3. Parameters to it.
4. Return result. At a minimum did it return success.
|
|
|
|
|
Wordle 838 4/6
⬛⬛⬛🟨⬛
⬛⬛🟩🟩🟩
⬛🟩🟩🟩🟩
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 838 4/6
⬜⬜⬜⬜🟨
⬜⬜🟨⬜⬜
⬜🟩🟩⬜⬜
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 838 3/6
⬜🟩⬜⬜⬜
⬜🟩🟩⬜⬜
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 838 3/6
⬜⬜⬜⬜⬜
🟨⬜⬜⬜🟨
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 838 6/6
⬜🟩⬜⬜⬜
⬜🟩⬜⬜⬜
⬜🟩🟩⬜⬜
⬜🟩🟩⬜⬜
⬜🟩🟩🟩🟩
🟩🟩🟩🟩🟩
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
|
|
|
|
|
Wordle 838 3/6
⬜🟩⬜🟨⬜
⬜🟩🟩🟩🟩
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 838 X/6*
⬜⬜⬜⬜⬜
🟨🟨🟨🟨⬜
⬜🟩🟩🟩🟩
⬜🟩🟩🟩🟩
⬜🟩🟩🟩🟩
⬜🟩🟩🟩🟩
Happiness will never come to those who fail to appreciate what they already have. -Anon
And those who were seen dancing were thought to be insane by those who could not hear the music. -Frederick Nietzsche
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
