Commit Graph

367 Commits

Author SHA1 Message Date
Matt Clay
650befed37 Add missing space after keywords. (#78480)
Also remove unnecessary parenthesis.
2022-08-08 13:18:28 -07:00
Matt Davis
0591efd5e6 remove Templar caching (#78419)
* source of templating bugs (and at least one CVE), only applies to single-var values anyway
2022-08-03 15:26:37 -07:00
Brian Coca
1f59bbf4f3 listify_lookup_plugin_terms deprecate dataloader (#78244)
* listify_lookup_plugin_terms deprecate dataloader

  deprecated useless dataloader pass to function
  also removed from callers in core


Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2022-07-12 16:10:32 -04:00
Matt Martz
18992b7947 Add new loop_control.extended_allitems option (#75760)
* Add new `loop_control.extended_allitems` option. Fixes #75216

* Add test for extended_allitems

* docs code block fix
2022-06-16 08:56:13 -05:00
Sloane Hertel
621e782ed0 Add toggle to fix module_defaults with module-as-redirected-action on a per-module basis (#77265)
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults

Add a toggle for action plugins to prefer the resolved module when loading module_defaults

Allow moving away from modules intercepted as actions pattern

Fixes #77059
2022-05-24 13:38:46 -04:00
Brian Coca
a0dede5458 ansible-connection verboistery (#77509)
* ansible-connection verboistery

  for cli, just use normal parser creation
 this also adds --help, but that seems fine
 also some error cleanup

Co-authored-by: Nathaniel Case <this.is@nathanielca.se>
2022-04-14 11:24:33 -04:00
Brian Coca
3b9592fcaf Move to display as single source of verbosity (#77498)
* Move to display as single source of verbosity

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2022-04-14 11:23:06 -04:00
Brian Coca
c9d3518d2f Fix final fact delegation (#77008)
* fix facts delegation loop overwrite

 partial revert of change to allow facts to be present in each loop iteration
 was not needed in final results as result processing alreayd had the disctiontion
 and ended up breaking the assumptions in the calling code.

 fixes #76676
2022-02-11 18:19:38 -05:00
Brian Coca
6d2d476113 avoid polluting vars with incorrect settings (#76590)
avoid polluting vars with incorrect settings

 simplify variables and templars
  - variables, original, only updated with final results, reset are copies of this
  - tempvars used for preliminary templating
  - cvars used for connection/shell/become plugins, delegation aware
  - vars_copy ignore tempvars updates and use connection plugin to get 
    finalized version per loop item/delegation,  also used to store temp results
    till we are ready to update 'variables'
 - fine tune nolog just cause we are here
 - also fix inventory_hostname_short for IP addresses
2022-02-07 16:09:28 -05:00
Brian Coca
be19863e44 ssh connection: use 'correct' host in all cases (#76017)
ssh plugin, use 'correct' information source in all cases
  * still fallback to pc
  * added inventory to new test
  * undef var can still show as parser error on pc
    now task_exectuer has a  more accurate error handling
2022-02-07 15:13:40 -05:00
Martin Krizek
394d216922 add_host/group_by: fix using changed_when in a loop (#71719)
Fixes #71627
Fixes #75971
2022-02-04 12:35:23 +01:00
Matt Martz
96ce4804ee Resolve perf issue with async callback events (#76783)
* Don't send full task with async callback events. Fixes #76729

* Use args for async_status task, instead of k=v

* Make sure we send back the async task attrs for polling

* Add clog frag

* load is a staticmethod
2022-01-21 11:47:32 -06:00
Brian Coca
7bec196061 loop/fact delegation fix (#75768)
now set_fact and include_vars intermediate results are congruent with delegation
2021-11-01 11:25:10 -04:00
Martin Krizek
0f95371131 Start of moving away from six (#75863)
ci_complete
2021-10-21 10:40:04 +02:00
Yedidyah Bar David
aff5e5dfc6 _run_loop: Add the task name to the warning (#76012)
Prepend the task name to the warning "The loop variable 'item' is
already in use", so that you can get some context even if stdout and
stderr go to separate places.
2021-10-14 10:31:04 -04:00
Sloane Hertel
3b861abce1 add action_groups support to collections (#74039)
* Canonicalize module_defaults actions and action_groups pre-fork and cache them on the play

* Call get_action_args_with_defaults with the resolved FQCN plugin and don't pass the redirect list

* Add validation for action_group metadata and a toggle to disable the warnings

* Handle groups recursively referring to each other

* Remove special-casing for non-fqcn actions in module_defaults groups

* Error for actions and groups in module_defaults that can't be resolved

* Error for fully templated module_defaults

* Add integration tests for action_groups

* Changelog
2021-07-14 13:33:28 -04:00
Sam Doran
a8de35e131 task_executor - use correct value for ssh connection retries (#75155)
Since the task and connection both have the same 'retries' keyword, the task default
would override the connection value.

Do not pass 'retries' from the task to the connection options.

* Set ssh_connection retries default value back to 0
  It was 0 before the move to config and was changed to 3 by accident.
2021-07-06 10:43:25 -04:00
David Shrewsbury
703cb79442 Implement async callbacks (#74953)
* add changelog and output from default callback
* add test
* add comments about TE task
2021-06-22 12:19:32 -04:00
Matt Martz
ffa548503d Wrap all results, regardless of register, except for actions with clean facts (#73161)
* Wrap all results, regardless of register, except for actions with clean facts. Fixes #21088

* ci_complete

* Add tests

* Add clog frag
2021-06-04 16:40:53 +02:00
Matt Martz
ac77911491 Prevent conditional templating errors from masking task output. Fixes #37187 (#74891) 2021-06-04 05:11:01 +10:00
Brian Coca
9c718ccc42 Async status rewrite
-  remove need for module (at least for posix side)
  - adds retry with backoff on fetching file, since
    race is bigger since we don't spend time on module
  - now gives more info on fail
  - also made actionfail/skip handle results if given
2021-06-03 09:54:40 -04:00
Brian Coca
b518aabf81 delegation fix (#74685)
*  delegation fix
2021-05-24 10:13:19 -04:00
Brian Coca
173d0a8de7 Skip interpreter discovery for network OSs(#74012)
skip python interpreter discovery on network os being set as this indicates a 'forced local' module and should use sys.executable.
2021-05-14 10:19:55 -04:00
Brian Coca
3cff54d69b Fix missing delegate display (#74370)
* dont rely on vars, task already gives us info
* ensure we always display delegation in host label
* also added parens with ansible_host to show  target host vs resolved host
* delegating to self is not delegating
* delegated vars restoration for backwards compat
* tests need mock task with delegate_to
2021-04-28 13:24:38 -04:00
Brian Coca
f9f839fa08 Fix debug factsetter (#74067)
* prevent debug from setting namespaced facts as tlv
* also added tests
2021-03-31 09:30:09 -04:00
Matt Martz
78f34786dd Send callbacks directly from the TaskExecutor instead of TaskResults masquerading as callbacks (#73927) 2021-03-18 12:12:29 -07:00
David Shrewsbury
78d3810fdf Auto cleanup of async cache file (#73760)
* Auto cleanup of async cache file

* Add changelog
2021-03-04 11:06:27 -08:00
Martin Krizek
7f9ac0f364 Consolidate filters/tests handling into JinjaPluginIntercept (#71463)
* Consolidate filters/tests handling into JinjaPluginIntercept

ci_complete

* Postpone loading all ansible plugins

* Do we need to create an overlay?

ci_complete

* Typo

ci_complete

* Add FIXME

* conditional.py: use public Environment.parse() method

* Remove remaining occurrences of shared_loader_obj being passed to Templar

* __UNROLLED__ not needed with this change anymore

* Incorrect rebase at some point?
2021-01-21 11:22:33 +01:00
Brian Coca
07248e5ec1 avoid key errors on environment access (#72620) 2020-11-17 12:09:46 -05:00
Ganesh B Nalawade
5fb6280672 Add verbose log for network action handler information 2020-11-05 11:11:22 -08:00
Felix Fontein
da60525610 Fix missing ansible.builtin FQCNs in hardcoded action names (#71824)
* Make sure hard-coded action names also check for FQCN.
* Use _add_internal_fqcn() to avoid hardcoded lists and typoes.
2020-11-03 08:51:31 -05:00
Matt Martz
be5fc4e642 Ensure we call action_loader.get with collection_list (#72206)
* Ensure we call action_loader.get with collection_list. Fixes #72170

* Add tests and changelog

* Remove grep, do assertion in playbook. ci_complete

* Skip old jinja2 versions

* ci_complete

* dedupe
2020-10-19 09:54:24 -05:00
Sloane Hertel
7048542199 Fix passing the connection timeout to connection plugins (#71722)
* Fix passing the connection timeout to connection plugins
2020-09-15 11:34:11 -04:00
Martin Krizek
d083307e36 Set global skipped result flag for looped tasks (#67847)
This allows for the skipped filter to be used on a registered looped
task results.

Fixes #16949
2020-08-28 08:01:26 +02:00
Martin Krizek
50f221c63e Fix incorrect msg in the results dict in loops (#71482) 2020-08-27 15:49:53 -04:00
Matt Martz
5821128995 Allow callbacks from forks (#70501)
* POC for supporting callback events that come from the worker

* linting fixes. ci_complete

* fix up units. ci_complete

* Try moving the sentinel put higher. ci_complete

* safeguards. ci_complete

* Move queue killing to terminate

* LINTING. ci_complete

* Subclass Queue, to add helper send_callback method

* Just use _final_q instead of adding another queue and thread

* Revert a few changes

* Add helper for inserting a TaskResult into the _final_q

* Add changelog fragment

* Address rebase issue

* ci_complete

* Add test to assert async poll callback from fork

* Don't use full path

* ci_complete

* Use _results_lock as a context manager

* Add new generic lock decorator, and use it with send_callback
2020-08-17 10:51:01 -05:00
Abhijeet Kasurde
504ef607f3 Misc typo fixes (#71089) 2020-08-05 10:16:44 +05:30
Brian Coca
f9af27c631 dont clobber facts in loop (#71032)
* dont clobber facts in loop

	fixes #71031
2020-08-04 15:41:42 -04:00
Jordan Borean
5e1a968983 Do not add connection vars to the output results (#70853)
* Do not add connection vars to the output results

* Also revert the delgated scenario JIC

* Added regression test
2020-07-24 09:53:08 +10:00
Matt Davis
4c0af6c808 fix internal cases of actions calling unqualified module names (#70818)
* fix internal cases of actions calling unqualified module names

* add porting_guide entry
* misc other fixes around action/module resolution broken by redirection

ci_complete

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* address review feedback

* pep8

* unit test fixes

* win fixes

* gather_facts fix module args ignores

* docs sanity

* pep8

* fix timeout test

* fix win name rewrites

Co-authored-by: Rick Elrod <rick@elrod.me>
2020-07-23 09:02:57 -07:00
Brian Coca
84adaba6f5 Allow hostvars delegation (#70331)
* ensure hostvars are available on delegation
* also inventory_hostname must point to current host and not delegated one
* fix get_connection since it was still mixing original host vars and delegated ones
* also return connection vars for delegation and non delegation alike
* add test to ensure we have expected usage when directly assigning for non delegated host
2020-07-22 11:13:57 -04:00
Abhijeet Kasurde
8915f262b4 misc typo fixes (#70736)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-19 18:51:29 -05:00
Sloane Hertel
8c213c9334 template connection variables accessed directly before using (#70657)
* template variables accessed directly when using them instead of FieldAttributes
2020-07-16 11:21:39 -04:00
David Shrewsbury
8aca464b8b Make sure ansible_become treated as a boolean (#70484)
* Make sure ansible_become treated as a boolean
2020-07-08 14:53:38 -04:00
Sloane Hertel
30e70f4b63 Handle post_validate templating errors and fix tests (#70240)
* Handle unexpected templating errors

* Fixes #70050

Fix up tests that weren't running and add tests for graceful templating error handling
2020-06-30 09:31:58 -04:00
Rick Elrod
9cfc76a221 Nuke with_ squashing, deprecated for 2.11 (#70209)
Change:
Removes with_* loop squashing and tests for 2.11

Test Plan:
CI, and grepped for with_items in package manager integration targets.
There might be some test cases in collections which need to stop testing
this behavior.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-06-23 11:53:25 -04:00
James Cassell
47d14a33bd config: singular ANSIBLE_COLLECTIONS_PATH (#70007)
* config: singular ANSIBLE_COLLECTIONS_PATH

Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories.  Let's be consistent both internally and with POSIX
tradition.

* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular

* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
2020-06-11 11:40:13 -07:00
Brian Coca
026bf4a9a8 do we really need update vars from pc? (#69952)
* don't we really need update vars from pc
2020-06-10 08:39:47 -04:00
Sloane Hertel
51f6d129cb support hard coded module_defaults.yml groups for collections (#69919)
* Only allow groups which were hardcoded in module_defaults.yml

only load action groups from the collection if module_defaults contains a potential group for the action

* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml

Third party modules should not be using group/ - use the action name instead

* add externalized module_defaults tests

add the missing group and collections

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

ci_complete

* Fix import in tests

ci_complete

* Update with requested changes

ci_complete

* don't traceback since we don't validate the contents of module_defaults

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>
2020-06-09 15:38:57 -07:00
Felix Fontein
a862ff2d43 Deprecation revisited (#69926)
* Allow to specify collection_name separately for deprecation.

* Use new functionality in Ansible.

* Use new functionality in tests.

* Update tagging/untagging functions.

* Update pylint deprecated sanity test.

* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).

* Improve version validation. Re-add version_added validation.

* Make sure collection names are added to return docs before schema validation.

* Extra checks to avoid crashes on bad data.

* Make C# module utils code work, and update/extend tests.

* Add changelog fragment.

* Stop extracting collection name from potentially tagged versions/dates.

* Simplify C# code.

* Update Windows modules docs.

* Forgot semicolons.
2020-06-09 15:21:19 -07:00