首页 \ 问答 \ 使用processing.Manager时对象是否重复?(Object duplicated when using processing.Manager?)

使用processing.Manager时对象是否重复?(Object duplicated when using processing.Manager?)

我正在尝试使用processing.Manager在进程之间共享数据(processing.Manager.list,processing.Manager.dict,...)。 但是,当我使用它时,对象是重复的:

from multiprocessing import Process, Manager


def manipulate_in_process(objects):
    objs = []
    for obj in objects:
        objs.append(obj)
    print('foo objs', objs)

def do_in_process(objects):
    print('objects', objects)
    manipulate_in_process(objects)

if __name__ == '__main__':
    manager = Manager()
    objects = [object(), object()]
    print('main objects', objects)
    shared_objects = manager.list(objects)
    process = Process(target=do_in_process, args=(shared_objects,))
    process.start()
    process.join()
    print('finish')

输出:

main objects [<object object at 0xb72a64a8>, <object object at 0xb72a64b0>]
objects [<object object at 0xb72a64a8>, <object object at 0xb72a64b0>]
foo objs [<object object at 0xb72a64e8>, <object object at 0xb72a64c0>]
finish

使用时共享对象重复: 0xb72a64a8 != 0xb72a64e8

  1. 我做错了?
  2. 为什么这些对象是重复的?
  3. 如何使用processing.Manager.list / dict / etc中的对象?

I'm trying to use processing.Manager to share data between processes (processing.Manager.list, processing.Manager.dict, ...). But, when i use it, object are duplicated:

from multiprocessing import Process, Manager


def manipulate_in_process(objects):
    objs = []
    for obj in objects:
        objs.append(obj)
    print('foo objs', objs)

def do_in_process(objects):
    print('objects', objects)
    manipulate_in_process(objects)

if __name__ == '__main__':
    manager = Manager()
    objects = [object(), object()]
    print('main objects', objects)
    shared_objects = manager.list(objects)
    process = Process(target=do_in_process, args=(shared_objects,))
    process.start()
    process.join()
    print('finish')

Output:

main objects [<object object at 0xb72a64a8>, <object object at 0xb72a64b0>]
objects [<object object at 0xb72a64a8>, <object object at 0xb72a64b0>]
foo objs [<object object at 0xb72a64e8>, <object object at 0xb72a64c0>]
finish

Shareds objects are duplicated when used: 0xb72a64a8 != 0xb72a64e8.

  1. I'm doing it wrong ?
  2. Why these objects are duplicated ?
  3. How can i use objects with processing.Manager.list/dict/etc ?

原文:https://stackoverflow.com/questions/27114266
更新时间:2019-12-07 10:49

最满意答案

在进程之间复制而不是共享对象几乎是固有的。 这是单独进程的定义:它们具有单独的堆与单独的对象。

如果您想要实际的共享数据,则必须使用ValueArraysharedctypes - 但在这种情况下,您只能使用“本机”值以及可以根据C结构表达的值。 而且,即使这样,也不能保证id在整个进程中是相同的,只是值本身就是这样。

Manager的要点是,您可以在服务器进程中放置一个对象,例如list ,并在每个子进程中放置该对象的代理。 就核心列表功能而言,列表代理看起来像一个真实的列表(迭代,索引, appendinsertindex等)。 但是代理并没有试图伪造像id这样的“隐藏”内省功能。

Manager可以直接代理的确切类型列表在Python版本之间有所不同,但您可以在您的版本的文档中找到它。 从3.4开始,它包括listdictNamespaceLockRLockSemaphoreBoundedSemaphoreConditionEventBarrierQueueValueArray 。 无论如何,它永远不会包含object ,因为object没有超出隐藏内容的功能。

当然,您可以使用自定义代理向Manager注册自己的类型。 但我不确定你想与object分享什么。

有关完整详细信息,请参阅概述的流程管理器 之间的共享状态


It's pretty much inherent that objects are copied, not shared, between processes. That's the definition of separate processes: that they have separate heaps with separate objects.

If you want actual shared data, you have to use Value, Array, or sharedctypes—but of course in that case, you're limited to "native" values and those you can express in terms of C structs. And, even then, there's no guarantee that the id will be the same across processes, only that the value itself will be.

The point of Manager is that you can put an object like, say, a list in a server process, and a proxy to that object in each child process. The list proxy looks like a real list as far as core list functionality goes (iteration, indexing, append, insert, index, etc.). But the proxy doesn't try to fake "under-the-covers" introspection functionality like id.

The exact list of types that can be proxied by Manager out of the box differs between Python versions, but you can find it in the docs for your version. As of 3.4, it includes list, dict, Namespace, Lock, RLock, Semaphore, BoundedSemaphore, Condition, Event, Barrier, Queue, Value and Array. At any rate, it's never going to include object, because object has no functionality beyond the under-the-covers stuff.

Of course you can register your own types with the Manager with custom proxies. But I'm not sure what you'd want to share with object.

See Sharing state between processes for an overview and Managers for full details.

2014-11-24

相关问答

更多

视图未在特定情况下绘制(View not painted in a particular case)

很抱歉迟到的回复,但请尝试禁用硬件加速。 一种方法是将以下内容添加到清单文件中: <application android:hardwareAccelerated="false" ...> Sorry for late response, but please try disabling hardware acceleration. One way to do that is to add the following to your manifest file: <application an

正则表达式匹配第一个和最后一个单词或任何单词(regex match first and last word or any word)

更新:根据@justhalf的建议,不需要使用正则表达式来分割单词。 如果要进行区分大小写匹配,请删除.lower() 。 这将匹配数据列表中的第一个单词和最后一个单词(不包括任何标点符号或尾随空格): (^\s?\w+\b|(\b\w+)[\.?!\s]*$) 火柴: MATCH 1-1. Terrific MATCH 2-1. Saturday. 2. Saturday MATCH 3-1. its MATCH 4-1. fabulous 2. fabulou

如何在基于类的视图中编写自己的方法并在url中调用方法(How to write own methods in class based views and call the methods in urls)

您需要在get() , post()或HTTP请求中调用方法。 没有启动HTTP请求,因此Django不会调用它。 您可以在视图中执行以下操作: def get(self, request, *args, **kwargs): return self.start() 每当您使用HTTP GET请求访问视图时,都会返回self.start()的结果。 You need to call methods in get(), post() or whatever the HTTP reques

如何在Windows Azure中获取Blob存储的签名?(How to get a Signature for a Blob Storage in Windows Azure ?)

使用共享密钥身份验证时,授权标头必须采用特定格式,并且必须能够使用匹配的签名对您的请求进行身份验证。 有关此主题的更多信息,请参阅我们的MSDN文章Windows Azure存储服务的身份验证 。 When using Shared Key authentication, the Authorization header must be in a specific format and must be able to authenticate your request with a matchi

Python - 列出问题(多个列表?)(Python - List issues (multiple lists?))

为什么不打开你的清单? >>> time = [0] >>> [time] = time >>> time 0 Why not just unpack your lists ? >>> time = [0] >>> [time] = time >>> time 0

从XP切换到Windows 7后,在Visual C#2008 Express Edition下出现“添加引用”问题(Problem with “add reference” under Visual C# 2008 Express Edition after switching from XP to Windows 7)

请原谅这个愚蠢的问题。 Have you installed MS Office since you switched from XP to windows 7? 我只是问,因为我忘了在同样的升级后安装它,上周我花了半个小时试图弄清楚为什么我用Powerpoint的项目抱怨。 I think I have got it. For the benefit of anyone who runs into the same problem, there was an interesting lesso

如何在MSSQL daba中保存'<'sysmbol从文本框中获取数据(How to save '<' sysmbol in MSSQL daba fetching data from the textbox)

使用参数化: - const string sql = "INSERT INTO EXAMS VALUES (@question, @op1, @op2, @op3, @op4, @ans)"; using (var cmd = new SqlCommand(sql, conn)) { cmd.Parameters.Add(new SqlParameter("@question", question.text)); cmd.Parameters.Add(new SqlParame

使用输入字符串作为函数名称c ++(Using an input string as a function name c++)

您可以使用非常简单的条件树: if (option == "Add") z = Add(ip1, ip2); else if (option == "Subtract") z = Subtract(ip1, ip2); else if (option == "Multiply") z = Multiply(ip1, ip2); else if (option == "Divide") z = Divide(ip1, ip2); 或者,您可以使用s

相关文章

更多

最新问答

更多
  • jsPlumb draggable element javascript函数(jsPlumb draggable element javascript function)
  • MVC4:ViewModel(带有radiobuttonlist)在HttpPost之后为空(MVC4: ViewModel (with radiobuttonlist) is empty after HttpPost)
  • 如何在同一帐户上设置“Dev repo”(在prod和团队之间)(How to set up a “Dev repo” (between the prod and the team) on the same account)
  • 如何在tcl中将eth0配置为发送方udp端口(how to configure eth0 as a sender udp port in tcl)
  • 在main方法中进行更改后传递给构造函数的TreeMap实例的行为是什么(What is the behavior of a TreeMap instance passed into a constructor following changes in main method)
  • 在这个的一些属性不同的颜色(Different color in some properties of this)
  • CURL没有返回任何内容(CURL Not returning anything)
  • MVC5注入依赖于城堡windsor的视图(MVC5 injecting dependency on a view with castle windsor)
  • CakePHP AJAX-Call:发生错误:未定义(CakePHP AJAX-Call: An error occured: undefined)
  • 如何用.aidl扩展名创建文件?(how to create a file with .aidl extension?)
  • 无法获得全尺寸动态WordPress灯箱图像显示(Cannot get full size dynamic WordPress lightbox image to display)
  • 使用Jsoup删除元素不起作用(Removing Element with Jsoup doesn't work)
  • 交叉编译ARM的MongoDB C ++驱动程序。(Cross compiling MongoDB C++ driver for ARM. Cannot find Boost Libraries)
  • ProgressDialog没有关闭(ProgressDialog not closing)
  • 单元测试DB中的JPA插入(Unit testing JPA insertion in the DB)
  • 点击谷歌服务对话框中的更新按钮(Click on update button in check google service dialog)
  • 适用于PhoneGap Build的iOS“缺少必需的57x57图标”(“Missing required 57x57 icon” for iOS on PhoneGap Build)
  • MVC ckeditor post编辑器值内联(MVC ckeditor post editor value inline)
  • Angular 2 Pipe - 无法读取未定义的属性'toString'(Angular 2 Pipe - Cannot read property 'toString' of undefined)
  • 安装引导加载程序(Installing bootloaders)
  • 用于确定最终如何包含头文件的工具(Tool to figure out how a header file is eventually being included)
  • 在将字符串转换为int [duplicate]之前检查字符串是否不是数字(Check if string is not a number before converting it to a int [duplicate])
  • 从脚本构建db2数据库以进行构建验证的工具?(Tools to build db2 database from scripts for build verification?)
  • 在偏移之后向FileOutputStream写一个数字(Write a number to FileOutputStream after an offset)
  • Javascript RegEx仅限数字(无特殊字符)(Javascript RegEx for Numbers Only (no special characters))
  • 我可以在PHP的Linux服务器上运行新的COM(“WScript.Shell”)(Can I run new COM(“WScript.Shell”) on linux servers in PHP)
  • 在Backbone.js中放置与视图不直接相关的代码(例如广告,分析等)的位置?(Where to place code not directly related to a view (such as ads, analytics etc) in Backbone.js?)
  • 适用于Mac和PC的一般开发人员必备软件综合列表[关闭](Comprehensive List of Essential Software for General Developers on Mac and PC [closed])
  • 将语言文本转换为模型(Transformation Language Text to Model)
  • 在控制器中的两个方法之间共享变量?(Sharing variables between two methods in a controller?)