Pandas Group by 2 列使用另一列查找增量

Question

我有一个 Pandas 数据框，它有 4909144 行，time作为索引source_name，dest_address, 和tvalue它与time索引相同。我已经整理由DF source_name，dest_address以及tvalue使用以下，使它们按时间分组，然后依次是：df = df.sort_values(by=['sourcehostname','destinationaddress','tvalue'])这给了我：                        source_name  dest_address   tvalue                 time                2019-02-06 15:00:54.000 source_1     72.21.215.90   2019-02-06 15:00:54.000 2019-02-06 15:01:00.000 source_1     72.21.215.90   2019-02-06 15:01:00.000 2019-02-06 15:30:51.000 source_1     72.21.215.90   2019-02-06 15:30:51.000 2019-02-06 15:30:51.000 source_1     72.21.215.90   2019-02-06 15:30:51.000 2019-02-06 15:00:54.000 source_1     131.107.0.89   2019-02-06 15:00:54.000 2019-02-06 15:01:14.000 source_1     131.107.0.89   2019-02-06 15:01:14.000 2019-02-06 15:03:02.000 source_2     69.63.191.1    2019-02-06 15:03:02.000 2019-02-06 15:08:02.000 source_2     69.63.191.1    2019-02-06 15:08:02.000 我想要时间之间的差异，所以我然后使用：#Create deltadf['delta'] = (df['tvalue']-df['tvalue'].shift()).fillna(0)这给了我：                        source_name  dest_address   tvalue                 deltatime                2019-02-06 15:00:54.000 source_1     72.21.215.90   2019-02-06 15:00:54.000 00:00:002019-02-06 15:01:00.000 source_1     72.21.215.90   2019-02-06 15:01:00.000 00:00:062019-02-06 15:30:51.000 source_1     72.21.215.90   2019-02-06 15:30:51.000 00:29:512019-02-06 15:30:51.000 source_1     72.21.215.90   2019-02-06 15:30:51.000 00:00:00但我想按source_nameand分组dest_address并获得差异，tvalue这样我就不会在第一个条目后遇到 delta喜欢-1 days +23:30:00或delta喜欢00:01:48的source_2时候应该是00:00:00。我在尝试：df.groupby(['sourcehostname','destinationaddress'])['tvalue'].diff().fillna(0)但这需要很长时间，并且可能无法为我提供我正在寻找的结果。

Answer 1

import datetime as dt

source_changed = df['sourcehostname'] != df['sourcehostname'].shift()

dest_changed = df['destinationaddress'] != df['destinationaddress'].shift()

change_occurred = (source_changed | dest_changed)

time_diff = df['tvalue'].diff()

now = dt.datetime.utcnow()

zero_delta = now - now

df['time_diff'] = time_diff

df['change_occurred'] = change_occurred

# Then do a function

# If df['change_occurred'] is True -> set the value of df['delta'] to zero_delta  

# Else set df['delta'] to the value at df['time_dff']