Are AI systems limited in their ability to learn/incorporate new information?

This past week Nature magazine published an article at https://doi.org/10.1038/s41586-024-07711-7  titled "Loss of Plasticity in Deep Continual Learning" by Shibhansh Dohare and others.

It basically claims that AI systems can not go on learning new things forever.

In contrast, the Human brain can continually learn and incorporate new information, over the whole of it's "defect free" lifetime.

The paper suggests a possible way of correcting this basic AI learning problem by resetting certain weights used in its neural network. This effectively introduces noise into the system.

Peter Brooks

Palm Bay Florida 

Parents
  • Hi Peter, Can we expect one AI system to be specialist in everything. In the realm of engineering, we often rely on specialists, whether it’s a structural engineer, a Electrical Consultant, or a electrician. Each brings deep expertise in their respective fields, contributing to more efficient and accurate problem solving. Could our AI models follow a similar path?

    - Andrew

  • Hello Andrew:

    It will go wherever there is the most "money" to be made.

    Each day I read numerous "medical" related articles in JAMA and NEJM and they are full of AI related articles some positive and many highly negative.

    Doctors don't like the idea of losing their power and position in society.  

    Peter Brooks

    Palm Bay

  • Hello Andrew:

    If you are interested in the latest "significant" world wide medical research, you might like to listen to the following weekly podcast:-

    podcasts.apple.com/us/podcast/tt-healthwatch/id914224242

    TTUHSC EL Paso Healthwatch

    Last Fridays edition contained details of two research papers which showed the proposed changes to medical practices/drugs "did not" result in any improvement in  patient outcomes.

    Test failures are sometimes more important than successes  

    Peter Brooks

    Palm Bay

  • Hi Peter, thanks for sharing the link. Do you believe there is a significant potential for job displacement with further AI innovation? Additionally, could AI of the near future be the solution to reducing GP waiting times? Anything seems to be possible, like you say with enough money thrown at it.


    - Andrew.

Reply
  • Hi Peter, thanks for sharing the link. Do you believe there is a significant potential for job displacement with further AI innovation? Additionally, could AI of the near future be the solution to reducing GP waiting times? Anything seems to be possible, like you say with enough money thrown at it.


    - Andrew.

Children
  • Hello Andrew:

    Yes I believe that increased AI use will result in significant job displacement over the next 5-10 years.

    We are already seeing that in Silicon Valley with IT software people being laid off.

    GP wait time is currently a local UK problem and not applicable here in the US, unless you are under retirement age and don't have the ability to pay for health insurance.

    The problem with AI systems being directly involved in one health decisions is that ones historical health data is incomplete. For example my GP (equivalent) does not receive information from Specialists in other areas (dental, eyes and hearing).

    A person who has type 2 diabetes may have significant eye problems and has to have special tests looking at the blood vessels at the back of the eye.

    For me I would not trust any AI generated cheat sheet printed out for my GP before my six monthly check up. Significant things can happen and do, since my last formal meeting.

    I don't know what you do, but I go into my GP office will a full listing of everything that has happened since our last meeting and a list of needs and discussion items.

    It tough to keep up, but it avoids communication problems.

    Concerning the developing situation with AI systems - the whole process is unsustainable - long term power usage and CPU development problems- example the latest Intel CPU stability problem.

    We will need new technology to reduce power, manufacturing costs and improved reliability.

    I have been waiting over the past 10 years for this new technology to appear- I know where it will come from but it needs some more breakthroughs.

    By the way I consider teaching coding to kids as useless unless it goes back to the silicon level and the resulting actual transistor level.

    Peter Brooks

    Palm Bay 

    P.S. Do you remember the hype over Blockchain a few years back? 

     

     

       

  • Hi Peter

    The current process for scheduling an appointment with my GP requires patients to call the GP’s office starting at 8 AM. Upon calling precisely at 8 AM, one often finds themselves already 20th in the queue. On some days, all appointments are fully booked, necessitating another attempt the following morning. Appointments cannot be made in person and must be scheduled via telephone. When connected to the receptionist, patients are asked to describe their symptoms, presumably for prioritization purposes. Typically, patients are then scheduled for a telephone consultation, often for the following week if fortunate. During this brief five-minute call, the doctor assesses the situation and prescribes medication, which can be collected from a pharmacy.
    I believe that this service could be efficiently managed using AI. Additionally, I have read that some hospitals utilize blockchain technology. My local hospital, however, still relies on paper-based systems, which I found to be highly inefficient during a recent visit with my son. After checking in at our local urgent care center, he was administered intravenous antibiotics. As he is 16 years old, he was classified as an adult and subsequently transferred to the hospital. Unfortunately, he spent the night on a bed in the corridor, while I remained seated on a plastic chair. The following morning at 11 AM, a doctor briefly examined him for five minutes, prescribed medication, and discharged him. I am convinced that AI could streamline this service, potentially eliminating the need to spend the night in a cold corridor, surrounded by intoxicated individuals and witnessing staff being assaulted, with my son’s bed being kicked by a disruptive neighbor, all while being packed in like sardines.

    - Andrew

  • Hello Andrew:

    Regarding your recent experience with the NHS system and your 16 year old son:- 

    Most families have at one time experienced, some if not all, of the events that you have described.

    It is stressful for all members of the family.

    It is also natural to feel helpless at such times.

    Being trained engineers we are always trying to find solutions. However many of these solutions require major additional funding or reorganization of the complete system. There is no magic bullet, such as using AI, in the short term. . 

    Use the events as a learning experience and ask yourself the following question- What can I do with my limited resources, to improve 

    the local situation?

    When it happened to my family I was already retired.

    I volunteered to help at one of our local hospitals a few days a week.

    One job I did was moving medical equipment (IV pumps, crash carts, feeding pumps from the sterilization department) to every department/ward area. Yes I was also dressed in medical scrubs while doing this job. 

    Another job was working in the heart center making sure the necessary supplies (examples masks, band aids) used by the nurses, were available in each patient room. Also provided warm blankets, toothbrushes and looked for missing false teeth. This was in addition to talking to the patients and their families if needed.

    I hope this example helps.

    Peter Brooks

    Palm Bay 

     

      

  • I apologise if it seems like I’m complaining; I understand there are far worse situations. What actions can I take? I primarily work as a design, installation, inspection, and test electrician, as well as an Authorized Person/Low Voltage (AP/LV) at a hospital. Regarding your initial post, could you explain  what is meant by “this noise into the system” ? I understand the concept of noise in electrical installations.

  • Hello Andrew:

    No need to apologize for anything. Life threw you a curve ball (as they say here in the US) and that is that.

    I just gave you an example of what I (and my wife) did under our conditions.

    For example volunteering in a hospital may not be a viable option in the UK. However it got me interested in Medical science. 

    Regarding adding noise in the system - that was a quote by the Guy who wrote the paper - it was made in a podcast the author also made.

    I believe he was saying, for AI systems one has to give the stable system in kick in the butt by changing some the lower level neuron weights, to get it started. In an analog system it would be like adding noise into the system.

    Peter Brooks

    Palm Bay 

      

  • Hi Peter,

    Perhaps Kirsten McCormick and Dr. Elise Tapping could provide a more detailed explanation regarding the introduction of noise into the AI system.


    - Andrew

  • Hello Andrew:

    I am a little surprised that they haven't already contributed to this discussion.

    I did find a web site that deals with this subject:-

    larksuite.com/en_us/topics/ai-glossary/noise 

    but it doesn't seem to suggest changing the weights.

    Peter Brooks

    Palm Bay