Errors in software (or spreadsheet) design tools

I am currently conducting research on error prevention strategies in engineering design software and would greatly appreciate your insights. The goal is to gather the aggregated responses, anonymise them, and share the findings in a research paper. This paper aims to highlight current shortcomings in our field and suggest methods for improvements.

My primary focus is understanding how various companies approach the auditing of both internal and external calculation tools.

  1. Audit Methods: Do you employ peer reviews, code audits, or self-assessments? Could you share which method is your go-to and why it is preferred?
  2. Audit Frequency: How often do you conduct these audits? I’ve observed that some companies only audit when a code change occurs, while others do so every time a major error is spotted. What is your company’s practice and why?
  3. External Tools: If you use external tools, how do you ensure their reliability? Do you conduct your own audits on the software, or do you rely on the supplier’s auditing system? If you do review their audit, what does that process look like and why?
  4. Access to Source Code: If you conduct an audit, how do you access the source code? If you don’t have access to the source code, what alternative methods do you use and why is that method acceptable?
  5. Raising Concerns: Lastly, how do you raise concerns about potential errors? More importantly, how do you validate that changes have been implemented effectively?

If you would prefer to private message me, please hover over my name and select 'Send Private Message' in the pop up window, or visit my full profile and then 'connect' - alternatively I have set up a google form which will not require your name, or email to populate your answers, and you will remain entirely anonymous: https://forms.gle/B8MkNGMqyEt2VhgK9

Thank you in advance for your time and insights. Your input will be invaluable to my research.

Parents
  • Hi Graham,

    Just a thought: you may want to create a method for users to submit their replies anonymously (or at least not publicly), respondents may not to want to advertise how they do this such that other organisations can use this as an argument to disparage their quality procedures.

    Also, it might encourage replies if you say what your research is for - are the aggregated responses going to be published for everyone's benefit? 

    For background, my perspective is as an ISA, so these are exactly the questions I ask of projects I'm assessing / auditing, and it can sometimes be quite challenging to get honest answers! There's the "right" answer, and then there's what the evidence shows actually happens... I notice that (except for question 1) you haven't put "why" in your questions, I find that's really useful to find out - two organisations can have very different practices (e.g. for audit frequency), and the "why" question can reveal that both are correct for different reasons, they've built different mitigations into their processes - or indeed face different situations.

    Sorry that none of that has answered your question! Except to add my two penn'orth that from my experience there is no "right" answer to any of these, it all depends on software complexity, maturity of the software, maturity of the team (i.e. how long that team have worked on that software), maturity of the organisation (i.e. how good / thorough / appropriate the software development procedures are), safety / reliability criticality., etc etc etc. I vividly remember the days when I was running two software "teams",  one of which was audited to death and in forensic detail, the other of which was, literally, one person coding in their bedroom on the other side of the world! But both were correct and appropriate for their respective application. 

    Hope you get some good responses, they are interesting and challenging questions.

    Thanks,

    Andy

  • Just a thought: you may want to create a method for users to submit their replies anonymously (or at least not publicly), respondents may not to want to advertise how they do this such that other organisations can use this as an argument to disparage their quality procedures.

    Good advice Andy Slight smile

      if you enable private messages to be sent by anyone with an account for EngX then those interested can PM you instead of replying publicly in the forum. 

    To PM another user, hover over their name and select Send Private Message in the pop up window, or click on their name to visit their full profile and then 'connect' from there. 

  • Hi Andy,

    Thank you for your thoughtful response. Your insights, especially from your unique perspective as an ISA, are incredibly valuable, and I appreciate the time you took to share them.

    I agree with your suggestion about allowing anonymous responses. I will amend my post so that people can email me privately.

    As for the purpose of my research, I apologise for not making it clear initially. The goal is to gather the aggregated responses, anonymise them, and share the findings in a research paper. This paper aims to highlight current shortcomings in our field and suggest methods for improvements.

    Your point about including “why” in the questions is well-taken. Understanding the reasons behind certain practices can indeed reveal that different approaches may be correct in their respective contexts. I’ll make sure to incorporate this aspect in my future inquiries.

    Thank you for your valuable input. It’s given me a lot to think about and will undoubtedly enhance my research.

    Best Regards,

    Graeme

  • Hi Lisa,

    Thank you for your response - this is a great idea, I will incorporate this into the post.

    Graeme

  • Hi Graham, be mindful of posting  your email address in this public forum though. That's why we encourage the use of the PM feature in EngX instead Slight smile

Reply
  • Hi Graham, be mindful of posting  your email address in this public forum though. That's why we encourage the use of the PM feature in EngX instead Slight smile

Children
No Data