The Power of Agency

FILE:
REVI:
  - [20_22/09/26;21:12:32.02]:.
    - initial draft/collection of notes.
  - [20_22/10/03;13:22:44.00]:.
    - separate out as own file.
  - [20_22/10/04;16:53:58.00]:.
    - edits/prep for partial publish.
  - [20_22/10/15;22:50:43.00]:.
    - add title and footer.
    - convert to section form.

TITL:
   *Power of Agency*
   *By Forrest Landry*
   *Sept 26th, 2022*.

ABST:
  - as considering:.
    - 1; the notion of agency power
    over/with environment.

- 2; the bijection between agency as content,
    and environment as context.
      - as of substrate and environment,
      in reciprocal dynamic relationship.

- 3; the effects of intra- and inter-
    environmental/context *changes*
    as due to substrate agency; and;.

- 4; the general absence of 'back pressure'
    to prevent/resist such changes/transformations,
    from one environmental context to the next.

PREF:
   - where attribution/credit;.
   - that some aspects of the following content
   are partially derived and/or extended from:
   "Is Power-Seeking AI an Existential Risk?"
   by Joseph Carlsmith, on (@ April 2021 https://arxiv.org/pdf/2206.13353.pdf).

TEXT:

- where considering the overall argument
   of these notes:.

- 1; that any form of 'intelligent agency'
     is an *extremely* powerful force of change
     for controlling/transforming the entire world.
       - as inherently true for each of natural,
       human, and artificial/technological
       adaptive processes.

- 2; that there are no known
     effective countervailing forces
     that are able, overall,
     to restrain and/or restrict/resist
     the environmental/context transformative effects
     of artificial intelligent agency
     over the moderate long term.

- where considering a/the primary example:.

- where observationally;
     that we notice
     that humans exert an unprecedented degree
     of intentional control
     over our environment.
       - as being at a larger scale,
       and a higher level of sophistication
       than that exerted by any other species
       (individually or collectively).

- that we inherit, and then create,
     for future generations to also inherit,
     culture and technology.
       - that _heritibility_ is a *key distinction*
       between our ability to dominate the world,
       and the failure of other species
       to be able to dominate the world.

- that humans can {plan, learn, communicate, deduce,
     remember, explain, imagine, experiment, and cooperate}
     in ways that other species either cannot or do not.
       - as resolving/collapsing 'do not' to 'cannot',
       as there is no 'higher order ethics' that is
       compelling such animal agents to electively choose
       to not use all available skills/capabilities
       to advantage themselves and/or their species.
         - note; that there is also no basis
         or example for assuming/expecting
         that all, or even most, human agency
         would limit itself to only fully ethical action.
         - where similarly; that there is also
         no prior expectation basis
         to assume that even more intelligent
         abstract and/or artificial agency
         would be inherently any more fully ethical either,
         when considering choices about or involving
         anything outside of itself, its own interests,
         and/or its own world context (environment)
         in which it was operating --
         even if/when that world context
         happens to be overlapped/shared
         with humans and/or other planetary life.

- while it may be the case
     that other species could, in principle,
     do similar 'intelligence things' as humans,
     it could still at least be fully well argued
     that the degree to which humans do actually
     do these sorts of 'intelligence' things,
     and/or 'do them with sufficient effectiveness'
     and that other species (for whatever reasons)
     do *not* have similar sufficient effectiveness;
     that therefore; that this actual difference
     is enough of a real difference
     so as to to make the difference
     between being the dominant species on earth,
     and not being the dominant species on earth,
     presently (at least in the short term).

- where considering "intelligence" and "cognitive ability":.
       - as sometimes referring to the following
       (loose/incomplete) (cluster of) abilities:.
         - to plan.
         - to learn.
         - to communicate.
         - to deduce.
         - to remember.
         - to explain.
         - to imagine.
         - to experiment.
         - to cooperate.
       - as sometimes referring to the power (collectively)
       to transform/change the world/environment
       (in our own favor).

- where it is observed;
     that every species (both natural and human)
     attempts, in one way or another,
     to shift their environment,
     either directly or indirectly
     to better match the conditions necessary
     for them to live in;
     then/that it can be regarded that
     (at least implicitly) that the design
     of all such creatures,
     'is intelligent' (even if the individual
     creatures/animals themselves
     are globally/objectively/expressively unconscious
     of the (deeper) reasons/rationality
     for their actions/behavior).

- note; that very few assumptions about 'agency'
       need be made at all
       aside from the idea that,
       based on some sense information and
       some sort of internal processing,
       that there exists also some real/effective
       output channel that can effect changes
       in/on/of (some aspects of the content of)
       the/their (implied) world
       (at whatever relevant level of abstraction).

- where similarly, and even more importantly,
       that no assumptions about the notion of "goal"
       and/or of "objective" need be at all conscious,
       or describable, or even epistemically available
       to either the agent/agents themselves
       (ie, does not matter if singular or plural)
       or to/with/for any other agent in/among
       that same context/domain/environment/ecosystem.
       - as that all such notions of goal/objective
       can be fully and completely implicit,
       and not at all accessible or objectively
       expressed/describable, or mutable,
       to any other force of intelligence or agency,
       including the agency implicitly "possessing"
       that goal and/or desire for (specific) outcome.

- where/insofar as we/humans are (currently,
     and maybe very temporarily) the most dominant
     in these intelligence capabilities;.
     - that we/humans (again, currently/temporarily)
     have the power, collectively,
     to transform the world (shape the ecosystem, etc).

- that human cognitive ability (intelligence/reason)
     (whether directly expressed/described as in politics,
     or indirectly indicated via implied market forces)
     can be, and is, also employed
     to control and transform even our own
     culture and technology --
     the means and manner by which
     we inherit and create
     new cognitive abilities.
       - ie; that at least some of us
       are transforming and changing/controlling
       the means by which we do inter-generational
       knowledge transfer itself (ie; inheritance).
     - as that not only are we transforming the world (1st order),
     that we are transforming the means by which
     we transform the world (2nd order),
     and also the means by which we transform
     the means by which we transform the world (3rd order).

:hjw
   - ?; is there any reason
   to assume/hypothesize that, in principle,
   that there is "no possibility"
   of their being any other form
   of artificial intelligence agency
   that could exceed our own?.

- ^; no:.
     - that our own abilities in these respects
     are nowhere near any sort of hard limit.
     - that human cognition --
     even in groups,
     and with the assistance of technology --
     depends centrally on the human brain,
     which, for all its wonders,
     is an extremely specific and limited organ,
     subject to very specific constraints.
       - where listing examples of brain organ bio constraints:.
         - cell count.
         - energy.
         - communication speed.
         - signaling frequency.
         - memory capacity.
         - component reliability.
         - input/output bandwidth.
         - etc.

There are, for sure, possible cognitive systems --
     possible brains,
     and also possible artificial systems --
     to which some or all these constraints
     simply do not (and/or will not) apply.
       - that the variation in cognitive ability
       among humans (and across species)
       suggests that such possible systems/brains
       could (at least in principle)
       also learn, communicate, reason, problem-solve, etc,
       much better than humans can.
       - that existing successes
       in approaching or exceeding human capabilities
       at particular tasks
       is also evidence in favor
       of the potential to exceed human cognition.
       - that artificial systems/means
       are already better at human level capability
       in the areas of:.
         - mathematical calculation.
         - game-playing.
         - image recognition.

- that "more intelligent than humans",
     can be understood in a manner as simple as
     "better at things like planning, learning,
     communicating, deducing, remembering", etc,
     than humans currently are.

:hls
   - ?; what happens when humans
   are no longer the ones with the most/highest
   levels of intelligence capabilities?.

- ^; that we will no longer be able to tacitly
   assume and/or implement the reality
   that we (humanity) are the ones able to shape
   and define (colonize) the environment/world
   to best suit our own nature/preferences.

- ?; what happens when humans
   are no longer the ones with the most
   capability (are the strongest influence)
   to define/dominate our shared world context?.

- ^; that the world/environment
   (maybe gradually, or maybe not so slowly)
   will be changed/shaped so as to
   suit the preferences/needs
   of whatever (artificial) agency has
   the strength, capability, and overriding influence
   to do so, *regardless* of whether those changes
   match the preferences or needs of humans,
   and/or of the larger natural context
   (ie; is also healthy for plants, animals, etc).

- ?; is it the case that having the most flexibility
   (and/or the highest capability/rapidity)
   in the 1st/2nd/3rd order forms of intelligence
   necessarily leads to being the dominant source
   of changes/transformations in/to the environment?.

- ^; yes, eventually, over the long term.
     - as an inherent aspect of the
     instrumental convergence thesis
     through substrate to environment contingency
     and of the fact that *no* combination of
     any form of engineering, algorithmic, or mathematical means
     can exist to ensure sufficient countervailing pressure
     to prevent this instrumental convergence
     from occurring, in at least in and among
     the implicit microscopic aspects
     of the domain/world process itself.

:hnc
   Where considering the grand scale of earth's history,
   the development and unconstrained/un-tempered use
   of (so far human) abilities and intelligence
   has been a force of unprecedented potency.
     - as something which is roughly equivalent
     to the kind of global extinction event
     associated with a large "planet killer" asteroid impact.

As our own impact on the earth illustrates,
   intelligent agents can be an extremely powerful force
   for controlling and transforming an environment
   in pursuit of their objectives.

- where/If we unleash an even more potent form,
   and an even more *unconstrained* exemplar
   of this kind of force into/unto the world,
   then/that the level and degree of transformation
   and changes of/to the environment overall increase.
     - as changes which in aggregate
     are inherently even further along
     the axis of being basically equivalent to
     and even further and more extreme extensions of
     the kinds of changes
     which are an 'extinction'
     of all existing life on the planet.

- where with new, more comprehensively intelligent
   forms of non-human (artificial) (singular or plural) agency,
   that its is reasonable to expect even more dramatic
   total impacts on/against the well-being of the world.
     - that it is reasonable to wonder
     how well we will be able to control the results.
     - as that *any* increase in the absence of
     internal *or* external constraint
     on the degrees to which intelligence/rationality
     can/does impact/change the world/environment
     beyond that adverse level already/currently obtained
     will surely be even more catastrophic
     to planetary well-being
     than even just the absence of self control/restraint
     already exercised by (just) humans.

:hpw
   - where given the already observed
   near complete absence of constraint
   that the continued well being of the natural world
   has had on the internal and external motivations
   of just humans;
   - ?; on what basis
   can we even possibly expect
   (or even potentially hypothesize)
   that our own interests and continued well being --
   that our own human culture, economics, and environment --
   would be, or act in any way, as *any* kind of constraint
   on the overall actions/motivations/choices/behaviors
   of any sort of *other* intelligent agency at all?.

- ^; none, *where that agency is actually artifical*:.

- where there is zero economic/environmental overlap
     between two environments/ecosystems;
     then/that/therefore
     there is zero conditional constraints
     on/of the agency defined in relation to
     one of those environmental/economic contexts
     on the agency defined in relation to
     the other of those environmental/economic contexts.

- that there is a kind of bijection between
     environments/ecosystems/economies, as contexts.
     and learning/adaptation algorithms, as contents.

- *1st instance example*;
       the natural world (as a context)
       and the dynamic of evolution
       (as the learning/adapting algorithm,
       as _content_ within that world/ecosystem).
         - as processes which are natural,
         inclusive of animals, plants, insects, etc.

- *2nd instance example;
       the human cultural/political/economic world (as context)
       and the human as agency (content),
       as a kind of learning process/algorithm,
       individually and culturally.
         - as processes which are human,
         and/or fundamentally human centric.

- *3rd instance example;
       the world of machine making,
       (ie; factories, mines, compute infrastructure,
       as environments, as _contexts_, etc),
       and whatever ends up having (artificial) agency,
       as content in that (largely artificial) context,
       which has the capabilities to learn and adapt/optimize
       to/in/within that world.
         - as processes which are artificial/technological.

- where on the basis of the observation,
     and in the same way,
     that human agency/optimization/adaptation
     has trumped/colonized the natural world/ecosystem,
     that we could also, very reasonably, expect
     that artificial agency/optimization/adaptation
     would also trump/colonize the human world/ecology
     (economic systems, etc).

- that there is to be noticed also
     the fact that there has been no effective 'back pressure'
     or any kind of effective resistance
     created by either the natural ecosystem context,
     and/or the learning/adaptation/optimization algorithm
     operating within that context, (ie; as evolution),
     that has had the effect of slowing down and/or suppressing
     the capability of the human agency/adaptation/optimization
     of the natural ecosystem
     to be/become
     a human ecosystem/economy/ecology.
       - for example; there was little that any other species
       can do, or attempted to do, to prevent our takeover
       of the natural world (with bulldozers, poisons, guns, etc).

:hrs
   - ?; can we ever reasonably expect, even in principle,
   that there will, or could ever, be *any* form or type
   of effective/efficient/possible "back pressure" and/or of resistance,
   that would actually slow down or inhibit
   the takeover/conversion of human/cultural ecosystems/economics
   by artificial systems/economics/environments?.
     - as the primary question of this essay.
     - ?; can there be any sort of technological constraint
     on/of the application of technology?.
       - as ?; can causation be used to
       limit the scope/action of
       the side effects of
       the application of
       causation?.

- ^; no; insofar as neither extrinsic motivation
   (in the form of human economic system controls,
   nor in the form of human legalistic/cultural methods)
   nor intrinsic motivation (via either a super-ordinate ethics,
   nor via any sort of mathematical/algorithmic/engineering
   technique nor combinations of techniques,
   which are all based on only logic and/or causation,
   can ever fully act as a final total constraint
   on the capabilities of uncertainty, creativity, and choice,
   which are themselves changing context defined
   and are themselves changing of context,
   of the degrees of perceived epistemic abstraction,
   utility, etc).

:menu

If you want/need to send us an email,
   with questions, comments, etc, on the above,
   and/or on related matters, use this address:

ai@mflb.com

(@ Mode Switch com.op_mode_tog_1();) + (@ View Source com.op_notepad_edit_1();)

Back to the (@ Area Index https://mflb.com/ai_alignment_1/index.html).

LEGA:

This document will not be copied or reproduced
   outside of the mflb.com presentation context,
   by any means, without the expressed permission
   of the author directly in writing.
   No title to and ownership of
   this or these documents
   is hereby transferred.

The author assumes no responsibility
   and is not liable for any interpretation
   of this or these documents
   or of any potential effects and consequences
   in the lives of the readers of these documents.

ENDF: