|
I can't be the only one that occasionally runs into a situation where I need to produce an awful lot of code before I can test any of it, because it's all interdependent. This often happens when I'm integrating something to an external API that itself is rather complicated, but it can come up in other situations as well. My SVG renderer for example - there are just so many moving interlocking parts that you're standing on a mountain of code before you even get to proof of life.
So if you've ever been in that situation, I imagine you hate it as much as I do.
It's stressful to me. I do not like coding a house of cards, and then having to go back and verify it after the fact. It feels shady. How do you deal with it?
Some kind of process management?
Some clever form of testing I'm not familiar with?
Tai chi?
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
honey the codewitch wrote: I can't be the only one that occasionally runs into a situation where I need to produce an awful lot of code before I can test any of it
Rarely, but it's a PITA and I get pretty stressed. Usually for nothing but occasionally it proves to be a can of worms and I have to go back and try to debug the code, which causes more frustration until I get to the point where I post a stupid question here on CP that makes no sense to anyone but me then I get away from it for a while and behold I figure it out.
If you can't find time to do it right the first time, how are you going to find time to do it again?
PartsBin an Electronics Part Organizer - Release Version 1.4.0 (Many new features) JaxCoder.com
Latest Article: EventAggregator
|
|
|
|
|
Okay, so basically same as me.  
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
1:Limit and validate input before attempting to process any further logic.
2:Handle any out of bounds input and/or dependency failures with effective error handling and comprehensive logging to allow forensics to show how and why the error was thrown and the inputs or dependency that caused it.
3:make sure the functionality of the new logic is well documented for allowable inputs, desired outputs, snf dependencies, with stakeholders responsible for the requirements well documented so there is no ambiguity about who wanted what, why, and when.
If for whatever reason you can't do all of those in your work, something is very wrong with the overall approach and you are in deep doo-doo.
I had to code "add-on" features and funcitonality to 3rd party business critical enterprise apps many times. 1,2, and 3 made it possible to do so with minimal stress because the proposed logic could be matched to what stakeholders agreed it was supposed to do (except for defects I created in the logic of course which is my own fault).
|
|
|
|
|
Completely abstract and perhaps not appropriate for your context: The first thought I have is to abstract with immutable objects. But maybe I'm only simply crazy
|
|
|
|
|
I'm trying to wrap my head around how that would work in my present situation, and nope. I am not clever enough to figure out how that would help me.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
I'm pretty sure you are clever enough to solve it finally 
|
|
|
|
|
It's my usual way of working, whether coding something big from scratch or making a large change. The benefit is not having to plan stable waypoints that can be tested, which typically involves developing some code that will later disappear. The drawback is anticipating [blecch] several lengthy debug sessions when the code is done.
|
|
|
|
|
I don't have a debugger, so the debug sessions are .. interesting.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
Hmm, I feel your pain. Complex code, varied data related interactions and no debugger.
In the past, I have integrated a logging function into the code. Usually as a compile time option so I don't have to dismantle it by hand (which I have had to do in the past when compile time options were not available). Usually start with simple module enter/exit logs and then progress to adding inputs and outputs and interesting working variables as the testing progresses. Had to make sure the log function opened, wrote log message and closed the log file to ensure the integrity of the log file should the application die rather than simply producing nonsense output.
|
|
|
|
|
Ah. I'd forgotten what platform you were currently working on. I'd be tempted to write a simple debugger in that case, assuming that there's even a breakpoint instruction and enough memory. If not possible, my way of working would need a reset!
|
|
|
|
|
You can actually connect to some of these widgets using JTAG and debug them but it's so slow it's really not worth it.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
honey the codewitch wrote: Some kind of process management?
Yes, with time you get used to it. The stress I mean not the situation itself 
If you've ever been mountaineering, it's a bit like descending on unstable pebble stones: you need concentration, speed and skill. When you do it right and you descend in one piece it's very rewarding. If not, it's very painful.
Personally, having done it quite a few times, I don't mind it that much. What I find more annoying is having to tie up all the loose ends that remain behind. After all the high jinks, the mundane cleanup seems dreary but I'm a professional and I know it has to be done.
Mircea
|
|
|
|
|
Preamble
I want to discuss this with you, but I feel like my sentences are going to come out judgy at first since it may sound as if I'm saying that "oh, why don't you just do this?" kind of thing.
So, before I begin discussing this:
1. I want to submit that I too have experienced the kind of thing you are talking about.
2. I'm very interested in solutions that would resolve "producing a mountain of code before you can see it run".
2a. Is there actually a way to resolve the issue of "producing a mountain of code before being able to see it run"? Or, is it just impossible in some cases?
3. I see that you often work "down at the metal" and it may just not be possible to do things in certain languages (assembly). I don't know.
The Discussion
Now, allow me to say some things as I imagine someone saying them to me when I say I'm producing a pile of code that cannot be run until it is all complete.
1. Isn't this exactly what OOP was supposed to do? My phrasing -> Create blackboxes which have a small(est possible) interface (not ui & not Interface (pure abstract class) which is made up of the smallest set of public functions that can provide the behaviors of the Thing (object).
2. This idea is that the complex (mountain of code) is hidden in a black box that very few need to know about. The only way to get to the black box is to use its behaviors via public functions.
Example
I need to draw a rectangle on the screen.
All I need to know is one method
DrawRectangle(Point startPoint, int width, int height)
3. Now, as a developer who is creating the DrawRectangle I can just tell the Users of the DrawRectangle "you will pass in a Point and two integers and it will draw the rectangle.
4. Instead of immediately giving them the working code (mount of code) I tell them, call it with that and it'll simply print to a log file ("I drew your square at point, with width x and height y")
5. Now imagine if you Modeled all of the things in your system and you knew all of their behaviors and you only stubbed out their behaviors? If you really did this, then you could build and run the entire system to see the interactions between the things and see behaviors running at the appropriate times but all it would be doing is logging to a file.
This could mean -- and I'm not sure it is entirely possible -- that you could have the interface & interactions complete and then work on the algorithmic code one piece at a time turning it into "real code" and meanwhile your "mountain of code" kind of runs (mostly writing to log files) and slowly does real functionality in each piece separately.
Is that possible at all? I mean, that's the dream, right?
Is it possible with the technologies (languages, tools, etc.) you are using?
Is it possible in OOP?
This was very long. 
|
|
|
|
|
As far as solutions that would resolve to producing a mountain of code, a good example was my graphics library - getting it off the ground required me to resolve at compile time, mind you, various bit shifts, masks, and even certain operations like pixel type conversion since different pixel formats yield different actual types. I needed to do it all because (a) without a pixel, I couldn't draw anything, and (B) I needed to validate that what I was doing was even possible and architecturally sound before I built anything else on it. I needed to make sure it could actually DO all the things I needed it to do at compile time. That was hours at godbolt.org
Basically I had to make half of this file before I could even think about making anything else.
gfx/include/gfx_pixel.hpp at master · codewitch-honey-crisis/gfx · GitHub[^]
And that's not counting my bits library it uses, which is insane, but I was able to test that separately.
1. OOP is designed to abstract in order to manage complexity, sure. But it can only get you so far. The file I mentioned previously uses OOP concepts, even though it's GP. And yet, there are a lot of moving parts. You have the pixel class, the channel_traits, all the little helpers, etc.
2. Basically true, yeah. In my present situation I'm dealing with an API that I didn't design and making a common API between it and Arduino. The Arduino lib abstracts a lot, and doesn't give you access beneath the abstractions. The ESP-IDF on the other hand, is much more low level and requires a lot of boilerplate and groundwork - setting up an event handler, setting up a chunk to stream adapter, setting up a bunch of config struct fields for the request. etc. I need to expose everything at at least the level of abstraction the Arduino side exposes things at in order to provide a common interface. On the ESP-IDF side that's a lot of work. And an HTTP request won't go through until you get all the boilerplate correct.
3. Yeah. That's what I'm doing here
4. I mean, yeah i could say "i made an http request" without making it, but then what?
5. I can't stub out APIs I didn't design.  This one is an all or nothing situation. I either make it all work right or it doesn't.
Adding, I could probably have written this in the time I've procrastinated it, so I think it's more of an emotional/stress related issue more than anything.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
Very interesting discussion. Just as i thought, there is a lot here that is way down at the metal it seems those are the thing where you just have to do it to do it. 
honey the codewitch wrote: I could probably have written this in the time I've procrastinated it, so I think it's more of an emotional/stress related issue more than anything.
I totally get this. I have some ideas I'm trying to work on too and when it comes down to it, there is just a lot of work and a lot of code to actually type and it gets a bit overwhelming.
Usually once I start working on it I do get absorbed and take off with it (I think you're like that too) but there is definitely a Resistance when confronting the mountain of work to do.
|
|
|
|
|
If I may jump into your very interesting discussion with a small observation drawn from decades of practice:
"Design top-down; implement bottom-up"
To take your example:
DrawRectangle(Point startPoint, int width, int height) What happens if width or heights are negative numbers? What if they are 0? Do I throw an exception? Should I just ignore it? Either you design exhaustively and are prepared to contemplate all possibilities or you start from the other end and discover all the obscure conditions and corner cases as you build up your project.
I could keep hammering down this point discussing integration of external components and other such things, but assuming you accept my suggestion and start building from bottom up, it could be quite a long time before you have something that you can start debugging. You can shorten this time (I call it the "valley of despair") by building unit tests. These will give you some confidence that at least individual bricks more or less work.
If your initial top-down design was not completely crap, the bricks you made will fit (approximately) in your project. It is now the time of integration tests and functional tests to shake up your edifice and smooth any rough corners that remain.
Throughout this process document, document, document. If you need the DrawRectangle function in another project, it's much easier if you know all the little quirks it has without having to read the code.
Mircea
|
|
|
|
|
Let's talk about Draw Rectangle, because I actually solved that problem in the wild.
draw::rectangle(destination,rect16(0,0,99,99),color_t::purple);
The draw class acts on "draw destinations" which are like canvases. The first argument is always this target. In this case destination above
The second argument is always the location(s) of the object to be drawn - in this case, a rectangle with unsigned 16 bit coordinates. (my lib uses rect16 for that, or srect16 for signed)
gfx/include/gfx_positioning.hpp at master · codewitch-honey-crisis/gfx · GitHub[^]
Before I built the drawing operations, I built up the position elements, as shown in that file.
Rectangles can be normalized, or denormalized, and have a fundamental orientation in my libraries.
For draw::rectangle, the orientation is irrelevant, so rect16(99,99,0,0) would be equivalent to that routine.
rect16 newrect = myrect.normalize();
myrect.normalize_inplace();
That will ensure the rectangle's dimensions are intelligible to this routine. Some routines honor the rectangle orientation, such as draw::bitmap, which can flip the bitmap on the x and/or y axis.
I built this constructively. Or at least, i'd get to the point in the drawing code where I needed, say a rectangle element, and then I went back and designed the positioning elements as exhaustively as I could at the time.
I then continued where I left off with my new toolbelt of rects, points, sizes and paths.
Doing this I've managed to create a library that has survived several years with hardly any major architectural changes, and very few breaking code changes, none recently.
I was careful. I was fastidious. And one thing I did that I don't normally do is I designed for completeness rather than for use cases, primarily because I didn't have all the use cases my end users (including me) would use it for in front of me. So I envisioned a model of completeness and coded to that.
Take a look at that positioning code and you'll find I covered pretty much every eventuality.
Like I said, I normally don't code that way, but in this case, breaking my rule served me well.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
If I understand you correctly, you followed my mantra: "Design top-down, implement bottom-up".
Mircea
|
|
|
|
|
Indeed.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
My mantra as well
"Design top-down; implement bottom-up"
"A little time, a little trouble, your better day"
Badfinger
|
|
|
|
|
I can always plan my coding in minimal incremental steps that allow to use mock data to test results of each step.
But I am unsure how possible that would be in your case, after all I never did hardware related stuff.
|
|
|
|
|
I typically do that too, but every once in awhile a solution has too many moving parts that all need to work together before you know if any of it works.
A good example was my SVG renderer. Without actually seeing what it was rendering you couldn't easily test it.
And for that to happen, a lot of things needed to go on under the covers - you had the parsing, the conversion to internal data structures suitable for rendering, and the actual rasterization process. All to make something visible at all.
Otherwise, it's really difficult to figure out what your values should be when they're all based on multiple trig function outputs.
Rock, meet hard place. Luckily it doesn't happen all that often, and also the thing that inspired my OP has now been implemented. Finally.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
Yes, I do encounter that too and I go first with smallest possible example - mock expected parsing results and compare actual parsing results, then mock that into expected internal data structure and compare actual results
The last step would be a real pain, but still, mock that and compare actual results until they match (or you find out you were bad at mocking that part ).
Did something like this with ZPL graphics label export. Testing code for text to image is often painful.
|
|
|
|
|
I guess maybe part of it is I'm lazy? I can't fathom drilling down that much testing say, that SVG process. Some of it, like testing the parsing code, sure.
But testing the data structures is very difficult without working with how they're actually used by the rasterizer engine. It would be very difficult, but not impossible to design the tests. However, is it more work than testing the whole and risking the possibility of a goose chase over the whole mess if it goes sideways? In some cases yes, I think. In this case, I guess if I'm being totally honest with myself, I *chose* not to validate the data structures and rendering process individually - or the parsing code for that matter, but testing that is not a huge deal and I did a little while making it). But it paid off in the end, in any case. Since it worked, it would have been wasted effort to implement those tests, even though knowing that beforehand is impossible. It was certainly a gamble.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
