I am studying Bach's MicroPsi cognitive architecture (Principles of synthetic intelligence, Oxford Univ. Press, 2009) and have downloaded a copy of the MicroPsi 2 software. MicroPsi has a heuristic value system that is modeled after that of humans. It is not based on a simple scalar utility, rather, it employs a set of drives and aversions that attempt to measure and respond to things like:
affiliation and external legitimacy
fulfillment of social norms
I have criticized the human value system. I believe that a system like MicroPsi's may make the AI more human but less rational. I don't think the values MicroPsi uses are the most fundamental values. (as defined by evolutionary biology, for example)
Like my Asa H MicroPsi "encodes semantic relationships in a hierarchical spreading activation network. The representations are grounded in sensors and actuators and are acquired by autonomous exploration."