|
Marc Clifton wrote: And said jobs are put into a workflow with predecessor / successor dependencies, such that if one of the .bat processes (which can be, say, an EXE or a SQL SP call) fails, the whole dependency chain fails?
Automated a build at a previous company, but that excludes the whole chain-fail thingy; you simply check the errorlevel-variable.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
We have enough Control-M batch jobs that we have a small team supporting them.
|
|
|
|
|
We have various 'scheduled tasks' that process customer uploaded files or perform sql backups/ftp functions. All of them use custom exes that I wrote. In the case of sql backups, the exe gets started with a job number that links to a text file with configuration info.
Why custom executables?
0: the tasks are much too complicated to handle with a batch file
1: security...if I'm decrypting/encrypting/password protecting stuff, I'd hate to have my sensitive info stored in a text file.
2: exes are what I know/deal with every day
3: easy to debug
"Go forth into the source" - Neal Morse
|
|
|
|
|
The key phrase for me is "at night". I have nightly batch files on each of our department servers that handle backups and other routine stuff. For me, this is an application of KISS.
Our automated build process OTOH is a Windows service written in C#. A base class executes all the steps required for the build, from retrieving source code from source control to archiving the build folder at the end in an .ISO file. Product-specific classes derived from the base class handle situations unique to each product. The process itself is fairly granular, and can be paused and resumed or canceled easily. If the service is running on one of our build servers it performs a production build. If it's running on a developer's machine, certain steps are omitted to reduce the build time.
Software Zen: delete this;
|
|
|
|
|
Batch files are most simple and can be run from scheduled task also and can send out emails, also you can call powershell it a bat file.
Caveat Emptor.
"Progress doesn't come from early risers – progress is made by lazy men looking for easier ways to do things." Lazarus Long
modified 6-Jun-18 15:02pm.
|
|
|
|
|
Marc Clifton wrote: Or does your company use a more sophisticated way of dealing with automating workflows and dependencies?
cronjobs running bash scripts count?
M.D.V. 
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
|
No, my current employer prefers to spend more and accomplish less.
modified 6-Jun-18 23:38pm.
|
|
|
|
|
One of the last places I worked at botched a mainframe migration to J2EE and runs both together. The J2EE works all day queuing data for batch processing. At night the (80's era) scheduler runs the jobs, and if one fails, everything fails. The jobs loads the data needed by the J2EE for the next day. It's a mess, and about half the IT staff is always on-call at night. Oh, and there's no notification system. One person must log in at about 3:00 am to see if everything worked. This is a government agent with a $1.1B budget!
|
|
|
|
|
Batch files? Absolutely, love 'em, love 'em, love 'em. Can't get my head around PowerShell.
This[^] is good, if you want '.bat++'. In fact, it offers a way superior command line to cmd.exe in general.
|
|
|
|
|
Using batch files here. And the batch code is actually pretty sophisticated (at least after I've took it upon me to make it work properly, the Linux guys who created that batch mess really didn't know what they're doing and by their own account, they don't want to know what they're doing).
|
|
|
|
|
I've used Hangfire in the past for scheduled database jobs, though I've not used any of the fancy workflow options available with it.
Hangfire Pro[^]
|
|
|
|
|
We run an overnight process that updates a database with information from a Finance system and this was done using a .NET exe file which is started by Windows Task Scheduler.
The exe file accesses the source DB in read only mode and the target DB in update mode. It runs each database update in the right order and provides log files and emailed error, progress and diagnostic reports.
I know that if the last report has been emailed then all is well otherwise where it fails is in the logs it leaves behind. It does occasionally fail but that is usually following a target db change that has not been fully tested.
|
|
|
|
|
You may check WexFlow or may I suggest you Jenkins ?
|
|
|
|
|
I also agree with using Jenkins and it is free
|
|
|
|
|
Our company uses Jenkins[^].
Builds can run over night, or be initiated at any time.
Jenkins can build out of GitHub, deploy to a VM (for eg.) and report success/failure results to Slack.
Cheers,
Mike Fidler
"I intend to live forever - so far, so good." Steven Wright
"I almost had a psychic girlfriend but she left me before we met." Also Steven Wright
"I'm addicted to placebos. I could quit, but it wouldn't matter." Steven Wright yet again.
|
|
|
|
|
batch files. Kind of. yes sort of. Usually powershell scripts with fairly good transaction processing roll back included that emails successes and failures.
We try to avoid the whole Rube Goldberg Machine setup where if one little thing goes wrong the whole contraption doesn't work.
It does seem to happen once in awhile. What really chafs me is when it is forced on you by the application vendor.
"Oh we have this timer service you setup to run this process. It needs to run this one first and then the second one. It usually takes 15 minutes except when it doesn't. It should be ran at night and no it doesn't email. you just have to manually check it. We recommend setting them to run 2 hours apart. No don't run anything else at that time. Opps sorry that latest patch caused the first one to run for 3 hours. Now everything is messed up. Yes you will need to rerun them during the day. Yes it will be down while this completes. sorry"
Yep, we have had that conversation once. Not fun.
To err is human to really mess up you need a computer
|
|
|
|
|
At work we use a product called "Cybermation" , its formal name is "Enterprise Scheduling System (EPS)". At first I hated it, then I thought, hey, it beats 5,000 tasks, or windows services.
A long term goal I have, is building something like it. The way it works with Active Directory is nice. It puts a small db on the target prod server, and credentials are cached.
In short, something like this is really beneficial, because we have 10's of thousands of jobs running on this.
Informatica is also funneled into this, as well as a lot of other big products.
Where there's smoke, there's a Blue Screen of death.
|
|
|
|
|
At work we use a product called "Cybermation" , its formal name is "Enterprise Scheduling System (EPS)". At first I hated it, then I thought, hey, it beats 5,000 tasks, or windows services.
A long term goal I have, is building something like it. The way it works with Active Directory is nice. It puts a small db on the target prod server, and credentials are cached.
In short, something like this is really beneficial, because we have 10's of thousands of jobs running on this.
Informatica is also funneled into this, as well as a lot of other big products.
Another Scheduling product I see i SKYBOT , it looks more modern.
Where there's smoke, there's a Blue Screen of death.
|
|
|
|
|
I tend to create SQL Server Agent jobs (C# console programs); with a scheduled start.
The "dashboard" works well. Jobs can be any type. Easily restarted.
No agent for the SQL Server Express version though.
"(I) am amazed to see myself here rather than there ... now rather than then".
― Blaise Pascal
|
|
|
|
|
Hi All,
Used a link on my bar in Chorme this am and got the hamsters eating boards error, tried going through Google and still had rodents eating PCB's. So I think the site could have been down... now its working...
|
|
|
|
|
"Site Bugs/Suggestions" 
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
I suppose so, but it's working now, my PC is known to have farts where things don't work, I suppose I should make more use of the is it me website...
|
|
|
|
|
It was for a while but, hey! Isn't the hamster picture one of the coolest error messages that you've ever seen?
98.4% of statistics are made up on the spot.
|
|
|
|
|
|
PeejayAdams wrote: Isn't the hamster picture one of the coolest error messages that you've ever seen?
Yes, but I prefer the 404 on CP mode...
Don't look for the page, just try to realize the truth... there is no page
with bob in kimono malforming a spoon.
M.D.V.
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
