Top products from r/agi

As an independent scholar who has created Artificial General Intelligence in both Forth for Robots and in Perl for Webservers, I am frequently having to "judge performance... when developing AGI" -- as you express it up above. I hope it helps to answer your (excellent) question if I describe here some of the ways in which I test each AGI program in the course of mind-design and mind-creation.

The most basic test of my AGI Minds is to let the AGI run without human input, to see if any bug crops up and to observe if the underlying conceptual and sensory arrays are properly recording and storing information. Of course, the no-input chain-of-thought test is entirely deterministic, because no random events are causing any variations in the behavior of the AGI. If the AGI does not run in a pre-determined way and there are indeed variations in its behavior, there must be some bug in the software. You must then troubleshoot and debug, but even the slightest change in the AGI software can have hidden consequences over many iterations of future mind-testing. If you do obtain the expected deterministic outputs from any new version of the AGI, it is helpful to save a file containing all the outputs and all the thinking of the AGI for future reference.

Once the AGI Mind seems to be running okay without human input, the next step in "benchmarking" or testing the AGI is to see how it responds to inputs of a very specific nature. I am currently testing the MindForth AGI code by inputting simple English sentences into the knowledge base (KB) of the AGI. When I input "robots know god" the Forthmind responds with a piece of knowledge about the concept involved -- "GOD DOES NOT PLAY DICE". That divine idea in the innate KB of the AGI serves several purposes. Its main purpose is to show how the AGI deals with the negation of verbs in sentences. It also serves to invite philosophic and theological discussion of the concept of a Supreme Being. Thirdly, it includes vocabulary useful for the testing of inferences as in "Boys play games" and "John is a boy" so as to generate the inferred question, "Does John play games?" Fourthly, it reminds Netizens of Albert Einstein, who famously said it first.

The input of "Robots know God" is not only a benchmark-test of mental association through the SpreadAct module, but also a test of the memory-retention of the toddler-level AGI Mind. Recently I have been satisfied to see that MindForth properly holds onto the input as an idea and somewhat later regurgitates the notion by saying "ROBOTS KNOW GOD". There is a time-lapse in between the initial input and the resurfacing of the idea in AGI consciousness because the AGI MindGrid inhibits the input idea and will not think of the idea again until the neural inhibition wears off.

There is currently a problem (I am working on it) with the input-test of "You know God". The AGI Mind converts the "you" concept to the internal "I" concept for self-referential thought, and in so doing loses track of the auditory memory of the word "you" -- on purpose, because the self-concept of "I" or "ego" must not be expressed during output with the pronoun "you". The AGI software instead searches for the memory-engram of a correct form of the ego-pronoun, such as "I" or "mine" or "me". The input of "You know God" was recently triggering the no-play-dice Einsteinian output but the idea was not re-surfacing in the output after plenty of time for the neural inhibition to wear off. A few days ago I tried to fix the problem, but I only made it worse. The AGI Forthmind started saying "ME AM ANDRU" over and over again. I have now described some of the ad-hoc, practical benchmark tests of a running AGI, and I am grateful for the opportunity to answer the question, especially since
recently in the AI subReddit there has been some serious misunderstanding of my sincere attempts to create a True AGI and to create whole new industries of Webserver AI-maintenance jobs for Perl programmers; of Robot AGI evolution jobs for Forth programmers; and of True AI chatbot-creation jobs for anyone so inclined.

I was going to reply to your conclusion, but instead I'm just going to recommend Rationality: AI to Zombies, as it says everything I have said and was going to say, but better and more exhaustively.