Instruction:
Implement a function in Python 3 to
read raw data from Siemens MRI scanners with IDEA VB15. Will return an array of measured k-space data from raw data from Siemens MRI scanners using IDEA VB15 (single value). If the option '-I' is used, then image space data will be returned instead. Usage: readMeasDataVB15 filename [ -t ] [ -rfft ] [ -r1 ] [ -rp ] [ -rn ] [ -skipts ] [ -nnavek ] [ -ros ] [ -rosa ] [ -I ] [ -w ] [-npz] Examples: python3 -m rawdatarinator.readMeasDataVB15 raw.dat -w or using the shortned alias... python3 -m rawdatarinator.raw raw.dat -w Command-line Options: filename Filename of file containing raw measurements. -rfft (resetFFTscale) Resets FFTscale and DataCorrection for each coil to 1. -r1 (readOneCoil) Read measurement data from from individual coil. -rp (readPhaseCorInfo) _ -rn (readNavigator) _ -skipts (skip readTimeStamp) _ -nnavek (nNavEK) _ -ros (removeOS) Flag to remove oversampling (OS) in the x direction. removeOS=True is more efficient as it processes each readout line independently, reducing the required memory space to keep all measured data. -rosa (removeOSafter) Flag to remove oversampling (OS) in the x direction. This works in image space, cutting FOV. Not likely a good idea for radial. -I (transformToImageSpace) Produce image space representation. Note that there is no correction for partial Fourier or parallel imaging k-space undersampling. The given version of code only uses numpy's FFT operation. -w (writeToFile) Save k-space or image space volume. Currently the output filename is auto generated. -npz (npz) Save k-space or image space volume using the .npz file extension. Default is to use hdf5 file standard. -h (help) Displays this documentation.

Response:
def readMeasDataVB15(filename,
                     resetFFTscale=False,
                     readOneCoil=False,
                     readPhaseCorInfo=False,
                     readNavigator=False,
                     readTimeStamp=True,
                     nNavEK=False,
                     removeOS=False,
                     removeOSafter=False,
                     transformToImageSpace=False,
                     writeToFile=False,
                     npz=False):
    """Read raw data from Siemens MRI scanners with IDEA VB15.

    Will return an array of measured k-space data from raw data from
    Siemens MRI scanners using IDEA VB15 (single value). If the option
    '-I' is used, then image space data will be returned instead.

    Usage:
    readMeasDataVB15 filename [ -t ] [ -rfft ] [ -r1 ] [ -rp ] [ -rn ]
                              [ -skipts ] [ -nnavek ] [ -ros ]
                              [ -rosa ] [ -I ] [ -w ] [-npz]

    Examples:
    python3 -m rawdatarinator.readMeasDataVB15 raw.dat -w

    or using the shortned alias...

    python3 -m rawdatarinator.raw raw.dat -w

    Command-line Options:
    filename        Filename of file containing raw measurements.

    -rfft (resetFFTscale)
                    Resets FFTscale and DataCorrection for each coil
                    to 1.

    -r1 (readOneCoil)
                    Read measurement data from from individual coil.

    -rp (readPhaseCorInfo)
                    _

    -rn (readNavigator)
                    _

    -skipts (skip readTimeStamp)
                    _

    -nnavek (nNavEK)
                    _

    -ros (removeOS)
                    Flag to remove oversampling (OS) in the x
                    direction. removeOS=True is more efficient as it
                    processes each readout line independently,
                    reducing the required memory space to keep all
                    measured data.

    -rosa (removeOSafter)
                    Flag to remove oversampling (OS) in the x
                    direction. This works in image space, cutting FOV.
                    Not likely a good idea for radial.

    -I (transformToImageSpace)
                    Produce image space representation. Note that
                    there is no correction for partial Fourier or
                    parallel imaging k-space undersampling.  The given
                    version of code only uses numpy's FFT operation.

    -w (writeToFile)
                    Save k-space or image space volume. Currently the
                    output filename is auto generated.

    -npz (npz)
                    Save k-space or image space volume using the .npz
                    file extension.  Default is to use hdf5 file
                    standard.

    -h (help)
                    Displays this documentation.
    """

    filename_temp = os.path.splitext(filename)[0]
    if transformToImageSpace is False:
        filenameOut = '%s_Kspace' % filename_temp
    else:
        filenameOut = '%s_imageSpace' % filename_temp

    # Useful Parameters
    globalHeader = 32
    localHeader = 128

    infoParser = InfoParser()
    xmlstr = infoParser.raw2xml(filename)

    # Start in MeasYaps. MeasYaps starts with <value mod="MeasYaps"> and ends at
    # the next XProtocol mod='Phoenix'
    startIdx = xmlstr.find('<value mod="MeasYaps">')
    endIdx = xmlstr.find('<XProtocol mod="Phoenix">')
    my_xmlstr = '<MeasYaps>' + xmlstr[startIdx:endIdx] + '</MeasYaps>' # add root

    # Parse into XML
    root = ET.fromstring(my_xmlstr)

    # Used to decode Partial Fourier fractions (poor man's switch)
    fractionDict = { 10: 1.0,
                     8 : 7./8.,
                     4 : 0.75,
                     2 : 5./8.,
                     1 : 0.5  }

    # vals are tuples: (key,search,default,lambda)
    #     key => key to the dictionary entry
    #     search  => string to search the xml document with
    #     default => if not found in xml doc, use this value
    #     lambda  => whatever value you end up with, operate on it with this
    #                anonymous function.
    vals = [
        ('ScanDimension','sKSpace.ucDimension',None,lambda x:int(x,16)),
        ('flag3D',None,None,lambda _:True if data['ScanDimension'] is 4 else False),
        ('NxAll','sKSpace.lBaseResolution',None,lambda x:int(x)),
        ('NyAll','sKSpace.lPhaseEncodingLines',None,lambda x:int(x)),
        ('OSfactorPE','sKSpace.dPhaseOversamplingForDialog',None,lambda x:1.0 if my_xmlstr.find('sKSpace.dPhaseOversamplingForDialog') < 0 else 1.0+float(x)),
        ('NzAll','sKSpace.lPartitions',None,lambda x:int(x)),
        ('OSfactor3D','sKSpace.dSliceOversamplingForDialog',None,lambda x:1.0 if my_xmlstr.find('sKSpace.dSliceOversamplingForDialog') < 0 else 1.0+float(x)),
        ('phaseResolution','sKSpace.dPhaseResolution',None,lambda x:float(x)),
        ('sliceResolution','sKSpace.dSliceResolution',None,lambda x:float(x)),
        ('Nsl','sSliceArray.lSize',None,lambda x:int(x)),
        ('Nconc','sSliceArray.lConc',None,lambda x:int(x)),
        ('Nc',None,None,lambda _:len(re.findall(r'\.lRxChannelConnected',my_xmlstr))),
        ('Nc',None,None,lambda _:data['Nc']-1 if my_xmlstr.find('AdjustSeq%/AdjCoilSensSeq') > 0 else data['Nc']),
        ('nContrast','lContrasts',1,lambda x:int(x)),
        ('nSet','lSets',1,lambda x:int(x)),
        ('nAverage','lAverages',1,lambda x:int(x)),
        ('nRepetition','lRepetitions',None,lambda x:1 if my_xmlstr.find('lRepetitions') < 0 else 1+int(x)),
        ('nPhase','sPhysioImaging.lPhases',1,lambda x:int(x)),
        ('fractionY','sKSpace.PhasePartialFourier',10,lambda x: fractionDict[int(x)]),
        ('fractionZ','sKSpace.SlicePartialFourier',10,lambda x: fractionDict[int(x)]),
        ('phasePartialFourierForSNR','sKSpace.dSeqPhasePartialFourierForSNR',1.0,lambda x:float(x)),
        ('EPIFactor','sFastImaging.lEPIFactor',None,lambda x:int(x)),
        ('turboFactor','sFastImaging.lTurboFactor',1,lambda x:int(x)),
        ('PATMode','sPat.ucPATMode',None,lambda x:int(x,16)),
        ('PATRefScanMode','sPat.ucRefScanMode',None,lambda x:int(x,16)),
        ('AccelFactorPE','sPat.lAccelFactPE',None,lambda x:int(x)),
        ('AccelFactor3D','sPat.lAccelFact3D',None,lambda x:int(x)),
        ('nRefLinesPE','sPat.lRefLinesPE',None,lambda x:0 if data['AccelFactorPE'] is 1 else int(x)),
        ('nRefLines3D','sPat.lRefLines3D',None,lambda x:0 if data['AccelFactor3D'] is 1 else int(x)) ]

    # Evaluate all tuples
    data = dict() # dictionary to store all values we want to compre with MATLAB
    for tup in vals:
        if tup[1] is not None:
            idx = my_xmlstr.find(tup[1])
        else:
            idx = -1

        if idx < 0:
            val = tup[2] # Take the default if we can't find it
        else:
            val = get_val_by_text(root,tup[1]).text

        afun = tup[3]
        if afun is not None:
            val = afun(val) # Evaluate anonymous function if provided
        data[tup[0]] = val # Store the value in the dictionary

    ## Now use the whole xml document
    root = ET.fromstring(xmlstr)

    # Enforce a max value for Nc
    Nct = get_yaps_by_name(root,'iMaxNoOfRxChannels',lambda x:int(x))
    if Nct is not None:
        data['Nct'] = Nct
        if Nct < data['Nc']:
            data['Nc'] = Nct

    # If this exists, then we'll need it
    nPhCorrScan = get_yaps_by_name(root,'lNoOfPhaseCorrScans',lambda x:int(x))
    if nPhCorrScan is not None:
        data['nPhCorrScan'] = nPhCorrScan

    # Define some more variables
    if data['turboFactor'] > 1:
        data['nPhCorEcho'] = 1
        data['nPhCorScan'] = 1
    if data['EPIFactor'] > 1:
        data['nPhCorScan'] = 1
        data['nPhCorEcho'] = 3

    if data['AccelFactorPE'] is 1:
        data['FirstFourierLine'] = 1
        data['FirstRefLine'] = 1
    else:
        data['FirstFourierLine'] = get_yaps_by_name(root,'lFirstFourierLine',lambda x:1+int(x),1)
        data['FirstRefLine'] = get_yaps_by_name(root,'lFirstRefLine',lambda x:1+int(x),1)

    if data['AccelFactor3D'] is 1:
        data['FirstFourierPar'] = 1
        data['FirstRefPartition'] = 1
    else:
        data['FirstFourierPartition'] = get_yaps_by_name(root,'lFirstFourierPartition',lambda x:1+int(x),1)
        data['FirstRefPartition'] = get_yaps_by_name(root,'lFirstRefPartition',lambda x:1+int(x),1)

    data['NxOS'] = get_yaps_by_name(root,'iNoOfFourierColumns',lambda x:int(x))
    data['OSfactorRO'] = 2 # we can actually find this in YAPS, but MATLAB impl gives as magic num
    data['Nx'] = int(np.around(data['NxOS']/data['OSfactorRO']))
    data['Ny'] = int(root.find("ParamFunctor[@name='FillMiniHeaderData']/ParamLong[@name='NoOfFourierLines']/value").text)
    data['Nz'] = get_yaps_by_name(root,'iNoOfFourierPartitions',lambda x:int(x))
    data['NxRecon'] = get_yaps_by_name(root,'iRoFTLength',lambda x:int(x))
    data['NyRecon'] = get_yaps_by_name(root,'iPEFTLength',lambda x:int(x))
    data['NzRecon'] = 1 if data['Nz'] is 1 else get_yaps_by_name(root,'i3DFTLength',lambda x:int(x))
    data['NslicePerConcat'] = get_yaps_by_name(root,'ushSlicePerConcat',lambda x:int(x))

    ## Partial Fourier Mode and Parameters
    data['PCAlgorithm'] = get_yaps_by_name(root,'lPCAlgorithm',lambda x:int(x))
    data['NoOfPhaseCorrColumns'] = get_yaps_by_name(root,'lNoOfPhaseCorrColumns',lambda x:int(x),data['Nx']/2)
    data['NoOfPhaseCorrLines'] = get_yaps_by_name(root,'lNoOfPhaseCorrLines',lambda x:int(x),data['NyAll']/2)
    data['NoOfPhaseCorrPartitions'] = get_yaps_by_name(root,'lNoOfPhaseCorrPartitions',lambda x:int(x),data['NzAll']/2)
    data['ColSlopeLength'] = get_yaps_by_name(root,'lColSlopeLength',lambda x:int(x),data['Nx']/4)
    # I think there's a mistake here in the MATLAB code: ColSlopeLength defined again instead of LinSlopeLength.
    # Commented is it how I think it should be and we'll go from there. The value is not actually used.
    # data['LinSlopeLength'] = get_yaps_by_name(root,'lLinSlopeLength',lambda x:int(x),data['NyAll']/4)
    data['ColSlopeLength'] = get_yaps_by_name(root,'lLinSlopeLength',lambda x:int(x),data['NyAll']/4)
    data['ParSlopeLength'] = get_yaps_by_name(root,'lParSlopeLength',lambda x:int(x),data['NzAll']/4)

    ## Raw data correction factors, use the MeasYaps portion of the xml document
    root = ET.fromstring(my_xmlstr)
    data['CorrFactor'] = np.ones(data['Nc'])
    for c in range(data['Nc']):
        text = 'axRawDataCorrectionFactor[0][%d].dRe' % c
        data['CorrFactor'][c] = 1 if my_xmlstr.find(text) < 0 else float(get_val_by_text(root,text).text)

        text = 'axRawDataCorrectionFactor[0][%d].dIm' % c
        if my_xmlstr.find(text) >= 0:
            data['CorrFactor'][c] = data['CorrFactor'][c] + 1j*float(get_val_by_text(root,text).text)

    ## FFT Correction Factors
    data['FFTCorrFactor'] = np.ones(data['Nc'])
    if resetFFTscale is False:
        data['FFTCorrFactor'] = np.ones(data['Nc'])
        for c in range(data['Nc']):
            text = 'asCoilSelectMeas[0].aFFT_SCALE[%d].flFactor' % c
            data['FFTCorrFactor'][c] = float(get_val_by_text(root,text).text)

    ## For PC Angio
    data['Nset'] = 1
    text = 'sAngio.ucPCFlowMode'
    data['PCMRAFlag'] = int(get_val_by_text(root,text).text,16) if my_xmlstr.find(text) > 0 else 0

    if data['PCMRAFlag'] is 1:
        text = 'sAngio.sFlowArray.lSize'
        if my_xmlstr.find(text) < 0:
            data['Nset'] = int(get_val_by_text(root,text).text)

    ## Recalculation of partial Fourier factors and EPI/turbo factor for EPI and TSE
    data['fractionPE'] = float(data['Ny'])/float(data['NyAll'])
    data['fraction3D'] = float(data['Nz'])/float(data['NzAll'])
    data['EPIFactor'] = np.around(data['fractionPE']*data['EPIFactor'])
    data['nEPITrain'] = data['Ny']/data['EPIFactor']
    data['turboFactor'] = np.around(data['fractionPE']*data['turboFactor'])
    data['nTSETrain'] = data['Ny']/data['turboFactor']
    data['Nc0'] = data['Nc']
    data['Nc'] = 1 if readOneCoil is True else data['Nc0']

    ## Calculation of the number of valid k-space readouts and k-space data matrix dimensions
    if data['PATMode'] is 1:
        data['nReadout'] = (
            data['nAverage']*    \
            data['nPhase']*      \
            data['nRepetition']* \
            data['nContrast']*   \
            data['Nsl']*         \
            data['Nz']*          \
            data['Nc']*          \
            data['Ny'])

    elif (data['PATMode'] is 2) and (data['PATRefScanMode'] is 2):
        if (data['Ny'] % 2) is 1:
            data['NyPAT'] = (data['Ny'] - 1 + data['nRefLinesPE']*(data['AccelFactorPE'] - 1))/data['AccelFactorPE']
        else:
            data['NyPAT'] = np.floor((data['Ny'] + data['nRefLinesPE']*(data['AccelFactorPE'] - 1))/data['AccelFactorPE'])

        data['nReadout'] = (
            data['nAverage']*    \
            data['nPhase']*      \
            data['nRepetition']* \
            data['nContrast']*   \
            data['Nsl']*         \
            data['Nz']*          \
            data['Nc']*          \
            data['NyPAT'])

    if removeOS is True:
        data['kSpace'] = np.zeros((
            data['nAverage'],
            data['nPhase'],
            data['nRepetition'],
            data['nContrast'],
            data['Nsl'],
            data['Nc'],
            data['Nz'],
            data['Nx'],
            data['Ny']), dtype=np.complex64)
    else:
        data['kSpace'] = np.zeros((
            data['nAverage'],
            data['nPhase'],
            data['nRepetition'],
            data['nContrast'],
            data['Nsl'],
            data['Nc'],
            data['Nz'],
            data['NxOS'],
            data['Ny']), dtype=np.complex64)

    if (readPhaseCorInfo is True) and (data['nPhCorScan'] > 0):
        data['kPhaseCor'] = np.zeros((
            data['nPhCorScan'],
            data['nPhCorEcho'],
            data['Nsl'],
            data['nRepetition'],
            data['Nc'],
            data['NxOS']), dtype=np.float32)

    if readNavigator is True:
        data['nNavigator'] = (
            data['nAverage']*    \
            data['nPhase']*      \
            data['nRepetition']* \
            data['nContrast']*   \
            data['Nsl']*         \
            data['Nz']*          \
            data['Nc']*          \
            data['nEPITrain']    \
            *data['nNavEK'])

        data['kNavigator'] = np.zeros((
            data['nAverage'],
            data['nPhase'],
            data['nRepetition'],
            data['nContrast']*nNavEK,
            data['Nsl'],
            data['Nc'],
            data['Nz'],
            data['nEPITrain'],
            data['NxOS']), dtype=np.float32)

    if readTimeStamp is True:
        data['nTimeStamp'] = (
            data['nAverage']*    \
            data['nPhase']*      \
            data['nRepetition']* \
            data['nContrast']*   \
            data['Nz'])

        data['timeStamp'] = np.zeros((
            data['nAverage'],
            data['nPhase'],
            data['nRepetition'],
            data['nContrast']*nNavEK,
            data['Nz']), dtype=np.float32)

    ## Data Readout and Reordering
    # Read k-space data
    data['noiseMeasCounter'] = 0
    data['noiseMeas'] = np.zeros((data['NxOS'],data['Nc']),dtype=np.complex64)
    data['navigatorPrep'] = 0
    data['LineN'] = -1
    data['Ndr'] = 1

    for r in range(data['Ndr']):
        data['xCoil'] = r
        with open(filename,'rb') as f:
            readFlag = True
            skipField = 0
            countNavigator = 0
            navigatorDataON = False

            temp1 = np.zeros(data['NxOS'])
            data['dataField'] = np.fromfile(f,dtype=np.int32,count=1)[0]
            f.seek(data['dataField'],os.SEEK_SET)

            while readFlag is True:
                if readTimeStamp is True:
                    f.seek(12,os.SEEK_CUR)
                    data['timeS'] = np.fromfile(f,dtype=np.uint32,count=1)[0]
                    f.seek(4,os.SEEK_CUR)
                else:
                    f.seek(20,os.SEEK_CUR)

                data['evalMask1'] = np.fromfile(f,dtype=np.uint32,count=1)[0]
                data['evalMask2'] = np.fromfile(f,dtype=np.uint32,count=1)[0]
                flag = [(32 - m.start()) for m in re.finditer('1',np.binary_repr(data['evalMask1'],32))]

                # Tuples: (key,dtype,afun)
                vals = [ ('Nxr',None,None),
                         ('Ncr',None,None),
                         ('Line',None,None),
                         ('Acquisition',None,None),
                         ('Slice',None,None),
                         ('Partition',None,None),
                         ('Echo',None,None),
                         ('Phase',None,None),
                         ('Repetition',None,None),
                         ('Set',None,None),
                         ('CutOffDataPre',None,lambda:f.seek(12,os.SEEK_CUR)),
                         ('CutOffDataPost',None,None),
                         ('KSpaceCentreColumn',None,None),
                         ('CoilMode',None,None),
                         ('ReadOutOffCentre',np.float32,None),
                         ('KSpaceCentreLineNo',None,lambda:f.seek(4,os.SEEK_CUR)),
                         ('KSpaceCentrePartitionNo',None,None),
                         ('Channel',None,lambda:f.seek(44,os.SEEK_CUR)) ]
                for tup in vals:
                    t = np.uint16 if tup[1] is None else tup[1]
                    if hasattr(tup[2], '__call__'):
                        tup[2]()
                    data[tup[0]] = np.fromfile(f,dtype=t,count=1)[0]

                f.seek(2,os.SEEK_CUR)

                if 1 in flag:
                    break

                if any([k for k in [2,22,26] if k in flag]):
                    if (22 in flag) and (readPhaseCorInfo is False):
                        skipField = data['nPhCorScan']*data['nPhCorEcho']*data['Ncr']*(localHeader + 8*data['Nxr']) - localHeader

                    if (22 in flag) and (readPhaseCorInfo is True):
                        skipField = -localHeader
                        f.seek(skipField,os.SEEK_CUR)
                        skipField = 0

                        for m in range(data['nPhCorScan']*data['nPhCorEcho']*data['Ncr']):
                            infoMDH_TimeStamp = readMDH_TimeStamp_VB13(f)
                            temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])

                            if 25 in flag:
                                temp[0::2] = np.flipud(temp[0::2])
                                temp[1::2] = np.flipud(temp[1::2])
                            if data['CutOffDataPre'] > 0:
                                temp[0:2*data['CutOffDataPre']] = 0
                            if data['CutOffDataPost'] > 0:
                                temp[len(temp) - 2*data['CutOffDataPost']:] = 0

                            data['kPhaseCor'][data['Echo'],np.ceil(m/data['Ncr']),data['Slice'],data['Repetition'],data['Channel'],:] = (temp[0::2] + 1j*temp[1::2]).astype(np.float32)

                    if (2 in flag) and (readNavigator is False):
                        skipField = data['Ncr']*(localHeader + 8*data['Nxr']) - localHeader
                    if (2 in flag) and (readNavigator is True):
                        if (countNavigator is False) and (navigatorPrep is False):
                            kNavigator = np.zeros((
                                data['Nxr'],
                                data['Ncr'],
                                data['nContrast']*nNavEK,
                                data['nEPITrain'],
                                data['Nz'],
                                data['Nsl'],
                                data['nAverage'],
                                data['nPhase'],
                                data['nRepetition']),dtype=np.float32)
                            kNavigatorTemp = np.zeros((
                                data['Nxr'],
                                data['Ncr'],
                                data['nContrast']*nNavEK),dtype=float32)
                            navigatorPrep = 1

                        skipField = -localHeader
                        f.seek(skipField,os.SEEK_CUR)
                        skipField = 0

                        for m in range(nNavEK*data['Ncr']):
                            infoMDH_TimeStamp = readMDH_TimeStamp_VB13(f)
                            temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])

                            if 25 in flag:
                                temp[0::2] = np.flipud(temp[0::2])
                                temp[1::2] = np.flipud(temp[1::2])
                            if data['CutOffDataPre'] > 0:
                                temp[0:2*data['CutOffDataPre']] = 0
                            if data['CutOffDataPost'] > 0:
                                temp[len(temp) - 2*data['CutOffDataPost']-1:] = 0;
                            kNavigatorTemp[:,data['Channel'],data['Echo']] = (temp[0::2] + 1j*temp[1::2]).astype(np.complex64)

                        navigatorDataON = True

                    if 26 in flag:
                        temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])

                        if 25 in flag:
                            temp[0::2] = np.flipud(temp[0::2])
                            temp[1::2] = np.flipud(temp[1::2])

                        data['noiseMeas'][:,data['Channel']] = temp[0::2] + 1j*temp[1::2]
                        skipField = 0

                    f.seek(skipField,os.SEEK_CUR)

                else:
                    temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])
                    if 25 in flag:
                        temp[0::2] = np.flipud(temp[0::2])
                        temp[1::2] = np.flipud(temp[1::2])
                    if data['CutOffDataPre'] > 0:
                        temp[0:2*data['CutOffDataPre']-1] = 0
                    if data['CutOffDataPost'] > 0:
                        temp[len(temp) - 2*data['CutOffDataPost']-1] = 0
                    if 11 in flag:
                        temp = data['CorrFactor'][data['Channel']]*temp
                    temp = data['FFTCorrFactor'][data['Channel']]*temp

                    if readOneCoil is False:
                        if removeOS is True:
                            temp1[len(temp1) - data['Nxr']:] = temp[0::2] + 1j*temp[1::2]
                            tempX = np.fftshift(np.fft(np.fftshift(temp1)))
                            tempK = np.fftshift(np.ifftshift(np.fftshift(tempX[np.around((data['NxOS'] - data['Nx'])/2):data['Nx'] + np.around((data['NxOS'] - data['Nx'])/2)])))
                            data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],data['Channel'],data['Partition'],:,data['Line']] = tempK.astype(np.complex64)
                        else:
                            data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],data['Channel'],data['Partition'],data['kSpace'].shape[7] - data['Nxr']:data['kSpace'].shape[7],data['Line']] = (temp[0::2] + 1j*temp[1::2]).astype(np.complex64)

                    elif (readOneCoil is True) and (data['Channel']+1 == coilIndex):
                        if removeOS is True:
                            temp1[len(temp1) - data['Nxr']:] = temp[0::2] + 1j*temp[1::2]
                            tempx = np.fftshift(np.fft(np.fftshift(temp1)))
                            tempK = np.fftshift(np.fft(np.fftshift(tempX[np.around((data['NxOS'] - data['Nx'])/2):data['Nx'] + np.around((data['NxOS'] - data['Nx'])/2)])))
                            data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],0,data['Partition'],:,data['Line']] = tempK.astype(np.complex64)
                        else:
                            data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],0,data['Partition'],data['kSpace'].shape[7] - data['Nxr']:,data['Line']] = (temp[0::2] + 1j*temp[1::2]).astype(np.complex64)

                    if (readTimeStamp is True) and (data['Channel'] == 0) and (navigatorDataON is True):
                        data['EPITrain'] = countNavigator % data['nEPITrain']
                        data['timeStamp'][data['Echo'],data['EPITrain'],data['Partition'],data['Slice'],data['Acquisition'],data['Phase'],data['Repetition']] = (0.0025*timeS).astype(np.complex64)

                    if (readNavigator is True) and (data['Channel'] == 0) and (navigatorDataON is True):
                        data['EPITrain'] = countNavigator % data['nEPITrain']
                        kNavigator[:,:,:,data['EPITrain'],data['Partition'],data['Slice'],data['Acquisition'],data['Phase'],data[Repetition]] = kNavigatorTemp.astype(np.complex64)
                        navigatorDataON = False
                        countNavigator += 1

                if 1 in flag:
                    break

    data['kSpace'] = np.squeeze(data['kSpace'])

    if len(data['kSpace'].shape) == 3:
        data['kSpace'] = np.transpose(data['kSpace'],[1,2,0])
    elif len(data['kSpace'].shape) == 4:
        if data['flag3D'] is False:
            data['kSpace'] = np.transpose(data['kSpace'],[2,3,0,1])
        else:
            data['kSpace'] = np.transpose(data['kSpace'],[2,3,1,0])
    elif len(data['kSpace'].shape) == 5:
        data['kSpace'] = np.transpose(data['kSpace'],[3,4,2,0,1])

    if transformToImageSpace is True:
        if data['flag3D'] is True:
            data['imSpace'] = np.fft.ifftshift(np.fft.ifftn(np.fft.fftshift(data['kSpace'],axes=(0,1,2)),axes=(0,1,2)),axes=(0,1,2))
            data['imSpace'][2] *= data['Nz']
        else:
            data['imSpace'] = np.fft.ifftshift(np.fft.ifft2(np.fft.fftshift(data['kSpace'],axes=(0,1)),axes=(0,1)),axes=(0,1))

        data['imSpace'][0] *= data['NxOS']
        data['imSpace'][1] *= data['Ny']

        if (removeOSafter is True) and (removeOS is False):
            if len(data['imSpace'].shape) == 2:
                data['imSpace'][0:data['NxOS']/4,:] = []
                data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4 :,:] = []
            elif len(data['imSpace'].shape) == 3:
                data['imSpace'][0:data['NxOS']/4,:,:] = []
                data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4:,:,:] = []
            elif len(data['imSpace'].shape) == 4:
                data['imSpace'][0:data['NxOS']/4,:,:,:] = []
                data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4:,:,:,:] = []
            elif len(data['imSpace'].shape) == 5:
                data['imSpace'][0:data['NxOS']/4,:,:,:,:] = []
                data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4:,:,:,:,:] = []

    if writeToFile is True:
        if transformToImageSpace is True:
            if npz:
                np.savez_compressed(filenameOut,imSpace=data['imSpace'],timeStamp=data['timeStamp'])
            else:
                with h5py.File('%s.hdf5' % filenameOut,'w') as f:
                    dset = f.create_dataset('kSpace',data=data['imSpace'])
        else:
            if npz:
                np.savez_compressed(filenameOut,kSpace=data['kSpace'],timeStamp=data['timeStamp'])
            else:
                with h5py.File('%s.hdf5' % filenameOut,'w') as f:
                    dset = f.create_dataset('kSpace',data=data['kSpace'])

    return(data)


Instruction:
How would you implement a function in Python 3 that
functions run when called from command line.

Response:
def main(args):
    '''Function run when called from command line.'''

    options = { '-rfft':   ['resetFFTscale',False],
                '-r1':     ['readOneCoil',False],
                '-rp':     ['readPhaseCorInfo',False],
                '-rn':     ['readNavigator',False],
                '-skipts': ['readTimeStamp',True],
                '-nnavek': ['nNavEK',False],
                '-ros':    ['removeOS',False],
                '-rosa':   ['removeOSafter',False],
                '-I':      ['transformToImageSpace',False],
                '-w':      ['writeToFile',False],
                '-npz':    ['npz',False] }

    decode_simple_opts(options,args,readMeasDataVB15)


Instruction:
Create a Python 3 function for
getting json keys from directory structure

Response:
def _get_keys_folder(jdir, key_path=None, in_memory=True,
                     ignore_prefix=('.', '_')):
    """ get json keys from directory structure

    e.g.

    jdir
        sub_dir1
            data1.json
            data2.json
        sub_dir2
            data.json

    _get_keys_folder(jdir)
    => ['sub_dir1', 'sub_dir2']
    _get_keys_folder(jdir,['sub_dir1'])
    => ['data1', 'data2']

    NB: json files are identified with .json extension
        files/directories beginning with '.' are ignored

    """
    if not hasattr(jdir, 'iterdir'):
        raise ValueError('jdir is not a path object; {}'.format(jdir))

    key_path = [] if key_path is None else key_path

    keys = []

    key_found = False if key_path else True
    search_key = key_path[0] if len(key_path) > 0 else None

    for jsub in jdir.iterdir():
        if jsub.is_file() and jsub.name[-5:] == '.json':

            name, ext = os.path.splitext(jsub.name)
            if name == search_key or not key_path:
                key_found = True
                if key_path:
                    return jkeys(jsub, key_path[1:], in_memory, ignore_prefix)
                else:
                    keys.append(name)

        elif (jsub.is_dir()
              and not jsub.name.startswith(ignore_prefix)
              and (jsub.name == search_key or not key_path)):

            key_found = True
            if jsub.name in keys:
                raise IOError(
                    'directory has a sub-dir and file with same name: '
                    '{1} and {1}.json in {0}'.format(jdir, jsub.name))
            if key_path:
                return jkeys(jsub, key_path[1:], in_memory, ignore_prefix)
            else:
                keys.append(jsub.name)

    if not key_found:
        raise KeyError('key not found: {0}'.format(search_key))

    return sorted(keys)


Instruction:
Write a Python 3 function for
getting keys for initial json level or at level after following key_path

Response:
def jkeys(jfile, key_path=None, in_memory=True, ignore_prefix=('.', '_')):
    """ get keys for initial json level, or at level after following key_path

    Parameters
    ----------
    jfile : str, file_like or path_like
        if str, must be existing file or folder,
        if file_like, must have 'read' method
        if path_like, must have 'iterdir' method (see pathlib.Path)
    key_path : list[str]
        a list of keys to index into the json before returning keys
    in_memory : bool
        if true reads json into memory before finding keys
        (this is faster but uses more memory)
    ignore_prefix : list[str]
        ignore folders beginning with these prefixes

    Examples
    --------

    >>> from jsonextended.utils import MockPath
    >>> file_obj = MockPath('test.json',is_file=True,
    ... content='''
    ... {
    ...  "a": 1,
    ...  "b": [1.1,2.1],
    ...  "c": {"d":"e","f":"g"}
    ... }
    ... ''')
    ...
    >>> jkeys(file_obj)
    ['a', 'b', 'c']

    >>> jkeys(file_obj,["c"])
    ['d', 'f']

    >>> from jsonextended.utils import get_test_path
    >>> path = get_test_path()
    >>> jkeys(path)
    ['dir1', 'dir2', 'dir3']

    >>> path = get_test_path()
    >>> jkeys(path, ['dir1','file1'], in_memory=True)
    ['initial', 'meta', 'optimised', 'units']

    """
    key_path = [] if key_path is None else key_path

    def eval_file(file_obj):
        if not in_memory:
            return _get_keys_ijson(file_obj, key_path)
        else:
            return _get_keys(file_obj, key_path)

    if isinstance(jfile, basestring):
        if not os.path.exists(jfile):
            raise IOError('jfile does not exist: {}'.format(jfile))
        if os.path.isdir(jfile):
            jpath = pathlib.Path(jfile)
            return _get_keys_folder(jpath, key_path, in_memory, ignore_prefix)
        else:
            with open(jfile, 'r') as file_obj:
                return eval_file(file_obj)
    elif hasattr(jfile, 'read'):
        return eval_file(jfile)
    elif hasattr(jfile, 'iterdir'):
        if jfile.is_file():
            with jfile.open('r') as file_obj:
                return eval_file(file_obj)
        else:
            return _get_keys_folder(jfile, key_path, in_memory, ignore_prefix)
    else:
        raise ValueError(
            'jfile should be a str, '
            'file_like or path_like object: {}'.format(jfile))


Instruction:
Can you implement a function in Python 3 that
reads json with keys

Response:
def _file_with_keys(file_obj, key_path=None, parse_decimal=False):
    """read json with keys

    Parameters
    ----------
    file_obj : object
        object with read method
    key_path : list[str]
        key to index befor parsing
    parse_decimal : bool
        whether to parse numbers as Decimal instances (retains exact precision)

    Notes
    -----
    ijson outputs decimals as Decimal class (for arbitrary precision)

    """
    key_path = [] if key_path is None else key_path

    try:
        objs = ijson.items(file_obj, '.'.join(key_path))
    except NameError:
        warnings.warn('ijson package not found in environment, \
        please install for on-disk key indexing', ImportWarning)
        data = json.load(
            file_obj, parse_float=Decimal if parse_decimal else float,
            object_hook=decode)
        return indexes(data, key_path)
    try:
        data = next(objs)  # .next()
    except StopIteration:
        raise KeyError('key path not available in json: {}'.format(key_path))

    # by default ijson parses Decimal values
    if not parse_decimal:
        convert_type(data, Decimal, float, in_place=True)

    datastr = json.dumps(data)
    data = json.loads(datastr, object_hook=decode)

    return data


Instruction:
Implement a function in Python 3 to
read in folder structure as json

Response:
def _folder_to_json(jdir, key_path=None, in_memory=True,
                    ignore_prefix=('.', '_'), dic={}, parse_decimal=False):
    """ read in folder structure as json

    e.g.

    jdir
        sub_dir1
            data.json
        sub_dir2
            data.json

    _folder_to_json(jdir)
    => {'sub_dir1':{'data':{...}},
        'sub_dir2':{'data':{...}}}

    NB: json files are identified with .json extension

    """
    key_path = [] if key_path is None else key_path

    if not hasattr(jdir, 'iterdir'):
        raise ValueError('jdir is not a path object; {}'.format(jdir))

    key_found = False if key_path else True
    search_key = key_path[0] if len(key_path) > 0 else None

    for jsub in jdir.iterdir():
        if jsub.is_file() and jsub.name.endswith('.json'):

            name, ext = os.path.splitext(jsub.name)
            if name == search_key or not key_path:
                key_found = True
                if key_path:
                    data = to_dict(jsub, key_path[1:], in_memory,
                                   ignore_prefix, parse_decimal)
                    if isinstance(data, dict):
                        dic.update(data)
                    else:
                        dic.update({_Terminus(): data})
                else:
                    dic[name] = to_dict(jsub, key_path[1:], in_memory,
                                        ignore_prefix, parse_decimal)

        elif (jsub.is_dir()
              and not jsub.name.startswith(ignore_prefix)
              and (jsub.name == search_key or not key_path)):

            key_found = True
            if jsub.name in dic.keys():
                raise IOError(
                    'directory has a sub-dir and file with same name: '
                    '{1} and {1}.json in {0}'.format(jdir, jsub.name))
            if key_path:
                sub_d = dic
            else:
                dic[jsub.name] = {}
                sub_d = dic[jsub.name]
            _folder_to_json(jsub, key_path[1:], in_memory, ignore_prefix,
                            sub_d, parse_decimal)

    if not key_found:
        raise KeyError('key not found: {0}'.format(search_key))


Instruction:
Write a Python 3 script to
input json to dict

Response:
def to_dict(jfile, key_path=None, in_memory=True,
            ignore_prefix=('.', '_'), parse_decimal=False):
    """ input json to dict

    Parameters
    ----------
    jfile : str, file_like or path_like
        if str, must be existing file or folder,
        if file_like, must have 'read' method
        if path_like, must have 'iterdir' method (see pathlib.Path)
    key_path : list[str]
        a list of keys to index into the json before parsing it
    in_memory : bool
        if true reads full json into memory before filtering keys
        (this is faster but uses more memory)
    ignore_prefix : list[str]
        ignore folders beginning with these prefixes
    parse_decimal : bool
        whether to parse numbers as Decimal instances (retains exact precision)

    Examples
    --------

    >>> from pprint import pformat

    >>> from jsonextended.utils import MockPath
    >>> file_obj = MockPath('test.json',is_file=True,
    ... content='''
    ... {
    ...  "a": 1,
    ...  "b": [1.1,2.1],
    ...  "c": {"d":"e"}
    ... }
    ... ''')
    ...

    >>> dstr = pformat(to_dict(file_obj))
    >>> print(dstr.replace("u'","'"))
    {'a': 1, 'b': [1.1, 2.1], 'c': {'d': 'e'}}

    >>> dstr = pformat(to_dict(file_obj,parse_decimal=True))
    >>> print(dstr.replace("u'","'"))
    {'a': 1, 'b': [Decimal('1.1'), Decimal('2.1')], 'c': {'d': 'e'}}

    >>> str(to_dict(file_obj,["c","d"]))
    'e'

    >>> from jsonextended.utils import get_test_path
    >>> path = get_test_path()
    >>> jdict1 = to_dict(path)
    >>> pprint(jdict1,depth=2)
    dir1:
      dir1_1: {...}
      file1: {...}
      file2: {...}
    dir2:
      file1: {...}
    dir3:

    >>> jdict2 = to_dict(path,['dir1','file1','initial'],in_memory=False)
    >>> pprint(jdict2,depth=1)
    crystallographic: {...}
    primitive: {...}

    """
    key_path = [] if key_path is None else key_path

    if isinstance(jfile, basestring):
        if not os.path.exists(jfile):
            raise IOError('jfile does not exist: {}'.format(jfile))
        if os.path.isdir(jfile):
            data = {}
            jpath = pathlib.Path(jfile)
            _folder_to_json(jpath, key_path[:], in_memory, ignore_prefix,
                            data, parse_decimal)
            if isinstance(list(data.keys())[0], _Terminus):
                data = data.values()[0]
        else:
            with open(jfile, 'r') as file_obj:
                if key_path and not in_memory:
                    data = _file_with_keys(file_obj, key_path, parse_decimal)
                elif key_path:
                    data = json.load(
                        file_obj, object_hook=decode,
                        parse_float=Decimal if parse_decimal else float)
                    data = indexes(data, key_path)
                else:
                    data = json.load(
                        file_obj, object_hook=decode,
                        parse_float=Decimal if parse_decimal else float)
    elif hasattr(jfile, 'read'):
        if key_path and not in_memory:
            data = _file_with_keys(jfile, key_path, parse_decimal)
        elif key_path:
            data = json.load(
                jfile, object_hook=decode,
                parse_float=Decimal if parse_decimal else float)
            data = indexes(data, key_path)
        else:
            data = json.load(
                jfile, object_hook=decode,
                parse_float=Decimal if parse_decimal else float)
    elif hasattr(jfile, 'iterdir'):
        if jfile.is_file():
            with jfile.open() as file_obj:
                if key_path and not in_memory:
                    data = _file_with_keys(file_obj, key_path, parse_decimal)
                elif key_path:
                    data = json.load(
                        file_obj, object_hook=decode,
                        parse_float=Decimal if parse_decimal else float)
                    data = indexes(data, key_path)
                else:
                    data = json.load(
                        file_obj, object_hook=decode,
                        parse_float=Decimal if parse_decimal else float)
        else:
            data = {}
            _folder_to_json(jfile, key_path[:], in_memory, ignore_prefix,
                            data, parse_decimal)
            if isinstance(list(data.keys())[0], _Terminus):
                data = data.values()[0]
    else:
        raise ValueError(
            'jfile should be a str, '
            'file_like or path_like object: {}'.format(jfile))

    return data


Instruction:
Implement a function in Python 3 to
turn generator sequence into sequence of chunks.

Response:
def chunks(items, chunksize):
    """Turn generator sequence into sequence of chunks."""

    items = iter(items)
    for first in items:
        chunk = chain((first,), islice(items, chunksize - 1))
        yield chunk
        deque(chunk, 0)


Instruction:
Can you tell what is the following Python 3 function doing
def unionfs(rw='rw', ro=None, union='union'):
    from functools import wraps

    def wrap_in_union_fs(func):
        """
        Function that wraps a given function inside the file system.

        Args:
            func: The function that needs to be wrapped inside the unions fs.
        Return:
            The file system with the function wrapped inside.
        """

        @wraps(func)
        def wrap_in_union_fs_func(project, *args, **kwargs):
            """
            Wrap the func in the UnionFS mount stack.

            We make sure that the mount points all exist and stack up the
            directories for the unionfs. All directories outside of the default
            build environment are tracked for deletion.
            """
            container = project.container
            if container is None or in_container():
                return func(project, *args, **kwargs)

            build_dir = local.path(project.builddir)
            LOG.debug("UnionFS - Project builddir: %s", project.builddir)
            if __unionfs_is_active(root=build_dir):
                LOG.debug(
                    "UnionFS already active in %s, nesting not supported.",
                    build_dir)
                return func(project, *args, **kwargs)

            ro_dir = local.path(container.local)
            rw_dir = build_dir / rw
            un_dir = build_dir / union
            LOG.debug("UnionFS - RW: %s", rw_dir)

            unionfs_cmd = __unionfs_set_up(ro_dir, rw_dir, un_dir)
            project_builddir_bak = project.builddir
            project.builddir = un_dir

            proc = unionfs_cmd.popen()
            while (not __unionfs_is_active(root=un_dir)) and \
                  (proc.poll() is None):
                pass

            ret = None
            if proc.poll() is None:
                try:
                    with local.cwd(un_dir):
                        ret = func(project, *args, **kwargs)
                finally:
                    project.builddir = project_builddir_bak

                    from signal import SIGINT
                    is_running = proc.poll() is None
                    while __unionfs_is_active(root=un_dir) and is_running:
                        try:
                            proc.send_signal(SIGINT)
                            proc.wait(timeout=3)
                        except subprocess.TimeoutExpired:
                            proc.kill()
                            is_running = False
                    LOG.debug("Unionfs shut down.")

            if __unionfs_is_active(root=un_dir):
                raise UnmountError()

            return ret

        return wrap_in_union_fs_func

    return wrap_in_union_fs

Response: This function is used to configure the UnionsFS feature.


Instruction:
How would you explain what the following Python 3 function does
def __update_cleanup_paths(new_path):
    cleanup_dirs = settings.CFG["cleanup_paths"].value
    cleanup_dirs = set(cleanup_dirs)
    cleanup_dirs.add(new_path)
    cleanup_dirs = list(cleanup_dirs)
    settings.CFG["cleanup_paths"] = cleanup_dirs

Response: Updates the list of paths to clean up afterwards.


Instruction:
Write a Python 3 function for
checking if a project lies outside of its expected directory.

Response:
def __is_outside_of_builddir(project, path_to_check):
    """Check if a project lies outside of its expected directory."""
    bdir = project.builddir
    cprefix = os.path.commonprefix([path_to_check, bdir])
    return cprefix != bdir


Instruction:
How would you explain what the following Python 3 function does
def __unionfs_set_up(ro_dir, rw_dir, mount_dir):
    mount_dir.mkdir()
    rw_dir.mkdir()
    if not ro_dir.exists():
        LOG.error("Base dir does not exist: '%s'", ro_dir)
        raise ValueError("Base directory does not exist")

    from benchbuild.utils.cmd import unionfs as unionfs_cmd
    LOG.debug("Mounting UnionFS on %s with RO:%s RW:%s", mount_dir, ro_dir,
              rw_dir)
    return unionfs_cmd["-f", "-o", "auto_unmount,allow_other,cow", rw_dir +
                       "=RW:" + ro_dir + "=RO", mount_dir]

Response: Setup a unionfs via unionfs - fuse.


Instruction:
Given the following Python 3 function, write the documentation
def get_precision_regex():
    expr = re.escape(PRECISION_FORMULA)
    expr += r'=\s*(\S*)\s.*\s([A-Z]*)'
    return re.compile(expr)

Response:
Build a regular expression used to extract precision
    metric from command output


Instruction:
Can you create a Python 3 function that
builds HPL data from basic parameters

Response:
def _build_data(self):
        """Build HPL data from basic parameters"""

        def baseN(nodes, mpn):
            return int(math.sqrt(mpn * 0.80 * nodes * 1024 * 1024 / 8))

        def nFromNb(baseN, nb):
            factor = int(baseN / nb)
            if factor % 2 != 0:
                factor -= 1
            return nb * factor

        def get_grid(nodes, ppn):
            cores = nodes * ppn
            sqrt = math.sqrt(cores)
            factors = [
                num for num in range(2, int(math.floor(sqrt) + 1)) if cores % num == 0
            ]
            if len(factors) == 0:
                factors = [1]

            diff = 0
            keep = 0
            for factor in factors:
                if diff == 0:
                    diff = cores - factor
                if keep == 0:
                    keep = factor
                tmp_diff = cores - factor
                if tmp_diff < diff:
                    diff = tmp_diff
                    keep = factor
            return [keep, int(cores / keep)]

        properties = dict(
            realN=nFromNb(baseN(self.nodes, self.memory_per_node), self.block_size),
            nb=self.block_size,
            pQ=get_grid(self.nodes, self.cores_per_node),
        )
        return self._data_from_jinja(**properties)


Instruction:
Can you generate the documentation for the following Python 3 function
def mpirun(self):
        cmd = self.attributes['mpirun']
        if cmd and cmd[0] != 'mpirun':
            cmd = ['mpirun']
        return [str(e) for e in cmd]

Response: Additional options passed as a list to the mpirun command


Instruction:
How would you implement a function in Python 3 that
performs variable substitution on the given string s.

Response:
def expandvars(s, vars=None):
    """Perform variable substitution on the given string

    Supported syntax:
    * $VARIABLE
    * ${VARIABLE}
    * ${#VARIABLE}
    * ${VARIABLE:-default}

    :param s: message to expand
    :type s: str

    :param vars: dictionary of variables. Default is ``os.environ``
    :type vars: dict

    :return: expanded string
    :rtype: str
    """
    tpl = TemplateWithDefaults(s)
    return tpl.substitute(vars or os.environ)


Instruction:
Can you implement a function in Python 3 that
queries an object and return True if all kwargs are True False otherwise.

Response:
def and_(cls, obj, **kwargs):
        """Query an object

        :param obj:
          object to test

        :param kwargs: query specified in kwargssql

        :return:
          `True` if all `kwargs` expression are `True`, `False` otherwise.
        :rtype: bool
        """
        return cls.__eval_seqexp(obj, operator.and_, **kwargs)


Instruction:
Can you generate the documentation for the following Python 3 function
def or_(cls, obj, **kwargs):
        return cls.__eval_seqexp(obj, operator.or_, **kwargs)

Response:
Query an object

        :param obj:
          object to test

        :param kwargs: query specified in kwargssql

        :return:
          `True` if at leat one `kwargs` expression is `True`,
          `False` otherwise.
        :rtype: bool


Instruction:
How would you explain what the following Python 3 function does
def xor(cls, obj, **kwargs):
        return cls.__eval_seqexp(obj, operator.xor, **kwargs)

Response:
Query an object.
        returns True if exactly one kwargssql
        is True otherwise False.


Instruction:
Explain what the following Python 3 code does
def __resolve_path(cls, obj, path):
        path = [p for p in path if p]
        if any(path):
            pathes = len(path)
            i = 0
            while i < pathes:
                # _get_obj_attr can supersede `i` because it might
                # evaluate the entire expression by itself.
                obj, i = cls._get_obj_attr(obj, path, i)
                i += 1
        else:
            raise Exception("Nothing to do")
        return obj

Response:
Follow a kwargsql expression starting from a given object
        and return the deduced object.


Instruction:
Can you generate the documentation for the following Python 3 function
def _sequence_map(cls, seq, path):
        if not any(path):
            # There is no further kwargsql expression
            return seq
        result = []
        for item in seq:
            try:
                result.append(cls.__resolve_path(item, path))
            except (KeyError, IndexError):
                pass
        return result

Response: Apply a kwargsql expression to every item of a sequence and returns it.


Instruction:
Create a Python 3 function to
return a function that returns True if the given operation is not met.

Response:
def _not(cls, operation):
        """not operation"""

        def _wrap(*args, **kwargs):
            return not operation(*args, **kwargs)

        return _wrap


Instruction:
Make a summary of the following Python 3 code
def populate(projects_to_filter=None, group=None):
    if projects_to_filter is None:
        projects_to_filter = []

    import benchbuild.projects as all_projects
    all_projects.discover()

    prjs = ProjectRegistry.projects
    if projects_to_filter:
        prjs = {}
        for filter_project in set(projects_to_filter):
            try:
                prjs.update({
                    x: y
                    for x, y in ProjectRegistry.projects.items(
                        prefix=filter_project)
                })
            except KeyError:
                pass

    if group:
        groupkeys = set(group)
        prjs = {
            name: cls
            for name, cls in prjs.items() if cls.GROUP in groupkeys
        }

    return {
        x: prjs[x]
        for x in prjs if prjs[x].DOMAIN != "debug" or x in projects_to_filter
    }

Response: Populate the list of projects that belong to this experiment.


Instruction:
Explain what the following Python 3 code does
def cc(project, detect_project=False):
    from benchbuild.utils import cmd

    cc_name = str(CFG["compiler"]["c"])
    wrap_cc(cc_name, compiler(cc_name), project, detect_project=detect_project)
    return cmd["./{}".format(cc_name)]

Response: Returns a clang that hides CFLAGS and LDFLAGS and returns a complete plumbum command to it.


Instruction:
Implement a function in Python 3 to
return a clang ++ that hides CFLAGS and LDFLAGS and returns a complete plumbum command to it.

Response:
def cxx(project, detect_project=False):
    """
    Return a clang++ that hides CFLAGS and LDFLAGS.

    This will generate a wrapper script in the current directory
    and return a complete plumbum command to it.

    Args:
        cflags: The CFLAGS we want to hide.
        ldflags: The LDFLAGS we want to hide.
        func (optional): A function that will be pickled alongside the compiler.
            It will be called before the actual compilation took place. This
            way you can intercept the compilation process with arbitrary python
            code.

    Returns (benchbuild.utils.cmd):
        Path to the new clang command.
    """
    from benchbuild.utils import cmd

    cxx_name = str(CFG["compiler"]["cxx"])
    wrap_cc(
        cxx_name, compiler(cxx_name), project, detect_project=detect_project)
    return cmd["./{name}".format(name=cxx_name)]


Instruction:
Explain what the following Python 3 code does
def compiler(name):
    pinfo = __get_paths()
    _compiler = local[name]
    _compiler = _compiler.setenv(
        PATH=pinfo["path"], LD_LIBRARY_PATH=pinfo["ld_library_path"])
    return _compiler

Response: Returns a usable clang ++ plumbum command that executes the specified clang ++ library


Instruction:
Can you generate the documentation for the following Python 3 function
def strip_path_prefix(ipath, prefix):
    if prefix is None:
        return ipath

    return ipath[len(prefix):] if ipath.startswith(prefix) else ipath

Response: Strip prefix from path.


Instruction:
Make a summary of the following Python 3 code
def unpickle(pickle_file):
    pickle = None
    with open(pickle_file, "rb") as pickle_f:
        pickle = dill.load(pickle_f)
    if not pickle:
        LOG.error("Could not load python object from file")
    return pickle

Response: Unpickle a python object from the given path.


Instruction:
Can you implement a function in Python 3 that
wraps the binary with the runtime extension of the project.

Response:
def wrap(name, project, sprefix=None, python=sys.executable):
    """ Wrap the binary :name: with the runtime extension of the project.

    This module generates a python tool that replaces :name:
    The function in runner only accepts the replaced binaries
    name as argument. We use the cloudpickle package to
    perform the serialization, make sure :runner: can be serialized
    with it and you're fine.

    Args:
        name: Binary we want to wrap
        project: The project that contains the runtime_extension we want
                 to run instead of the binary.

    Returns:
        A plumbum command, ready to launch.
    """
    env = __create_jinja_env()
    template = env.get_template('run_static.py.inc')

    name_absolute = os.path.abspath(name)
    real_f = name_absolute + PROJECT_BIN_F_EXT
    if sprefix:
        run(uchroot()["/bin/mv",
                      strip_path_prefix(name_absolute, sprefix),
                      strip_path_prefix(real_f, sprefix)])
    else:
        run(mv[name_absolute, real_f])

    project_file = persist(project, suffix=".project")

    env = CFG['env'].value

    bin_path = list_to_path(env.get('PATH', []))
    bin_path = list_to_path([bin_path, os.environ["PATH"]])

    bin_lib_path = list_to_path(env.get('LD_LIBRARY_PATH', []))
    bin_lib_path = list_to_path([bin_lib_path, os.environ["LD_LIBRARY_PATH"]])

    with open(name_absolute, 'w') as wrapper:
        wrapper.write(
            template.render(
                runf=strip_path_prefix(real_f, sprefix),
                project_file=strip_path_prefix(project_file, sprefix),
                path=str(bin_path),
                ld_library_path=str(bin_lib_path),
                python=python,
            ))

    run(chmod["+x", name_absolute])
    return local[name_absolute]


Instruction:
Can you generate the documentation for the following Python 3 function
def wrap_cc(filepath,
            compiler,
            project,
            python=sys.executable,
            detect_project=False):
    """
    Substitute a compiler with a script that hides CFLAGS & LDFLAGS.

    This will generate a wrapper script in the current directory
    and return a complete plumbum command to it.

    Args:
        filepath (str): Path to the wrapper script.
        compiler (benchbuild.utils.cmd):
            Real compiler command we should call in the script.
        project (benchbuild.project.Project):
            The project this compiler will be for.
        python (str): Path to the python interpreter we should use.
        detect_project: Should we enable project detection or not.

    Returns (benchbuild.utils.cmd):
        Command of the new compiler we can call.
    """
    env = __create_jinja_env()
    template = env.get_template('run_compiler.py.inc')

    cc_fname = local.path(filepath).with_suffix(".benchbuild.cc", depth=0)
    cc_f = persist(compiler, filename=cc_fname)

    project_file = persist(project, suffix=".project")

    with open(filepath, 'w') as wrapper:
        wrapper.write(
            template.render(
                cc_f=cc_f,
                project_file=project_file,
                python=python,
                detect_project=detect_project))

    chmod("+x", filepath)
    LOG.debug("Placed wrapper in: %s for compiler %s", local.path(filepath),
              str(compiler))
    LOG.debug("Placed project in: %s", local.path(project_file))
    LOG.debug("Placed compiler command in: %s", local.path(cc_f))
    return local[filepath]

Response: Wrap a C ++ compiler command in a new file.


Instruction:
Can you tell what is the following Python 3 function doing
def persist(id_obj, filename=None, suffix=None):
    if suffix is None:
        suffix = ".pickle"
    if hasattr(id_obj, 'id'):
        ident = id_obj.id
    else:
        ident = str(id(id_obj))

    if filename is None:
        filename = "{obj_id}{suffix}".format(obj_id=ident, suffix=suffix)

    with open(filename, 'wb') as obj_file:
        dill.dump(id_obj, obj_file)
    return os.path.abspath(filename)

Response: Persist an object in the filesystem.


Instruction:
Here you have a function in Python 3, explain what it does
def load(filename):
    if not os.path.exists(filename):
        LOG.error("load object - File '%s' does not exist.", filename)
        return None

    obj = None
    with open(filename, 'rb') as obj_file:
        obj = dill.load(obj_file)
    return obj

Response: Load a pickled object from the filesystem.


Instruction:
Given the following Python 3 function, write the documentation
def extend_shapes(df_shapes, axis, distance):
    '''
    Extend shape/polygon outline away from polygon center point by absolute
    distance.
    '''
    df_shapes_i = df_shapes.copy()
    offsets = df_shapes_i[axis + '_center_offset'].copy()
    offsets[offsets < 0] -= distance
    offsets[offsets >= 0] += distance
    df_shapes_i[axis] = df_shapes_i[axis + '_center'] + offsets
    return df_shapes_i

Response: Extend shape or polygon outline away from polygon center point by absolute distance.


Instruction:
Implement a Python 3 function for
returning a list of lists of adjacent electrodes in a single table.

Response:
def extract_adjacent_shapes(df_shapes, shape_i_column, extend=.5):
    '''
    Generate list of connections between "adjacent" polygon shapes based on
    geometrical "closeness".

    Parameters
    ----------
    df_shapes : pandas.DataFrame
        Table of polygon shape vertices (one row per vertex).

        Table rows with the same value in the :data:`shape_i_column` column
        are grouped together as a polygon.
    shape_i_column : str or list[str]
        Column name(s) that identify the polygon each row belongs to.
    extend : float, optional
        Extend ``x``/``y`` coords by the specified number of absolute units
        from the center point of each polygon.
        Each polygon is stretched independently in the ``x`` and ``y`` direction.
        In each direction, a polygon considered adjacent to all polygons that
        are overlapped by the extended shape.

    Returns
    -------
    pandas.DataFrame
        Adjacency list as a frame containing the columns ``source`` and
        ``target``.

        The ``source`` and ``target`` of each adjacency connection is ordered
        such that the ``source`` is less than the ``target``.
    '''
    # Find corners of each solid shape outline.
    # Extend x coords by abs units
    df_scaled_x = extend_shapes(df_shapes, 'x', extend)
    # Extend y coords by abs units
    df_scaled_y = extend_shapes(df_shapes, 'y', extend)

    df_corners = df_shapes.groupby(shape_i_column).agg({'x': ['min', 'max'],
                                                        'y': ['min', 'max']})

    # Find adjacent electrodes
    row_list = []

    for shapeNumber in df_shapes[shape_i_column].drop_duplicates():
        df_stretched = df_scaled_x[df_scaled_x[shape_i_column]
                                   .isin([shapeNumber])]
        xmin_x, xmax_x, ymin_x, ymax_x = (df_stretched.x.min(),
                                          df_stretched.x.max(),
                                          df_stretched.y.min(),
                                          df_stretched.y.max())
        df_stretched = df_scaled_y[df_scaled_y[shape_i_column]
                                   .isin([shapeNumber])]
        xmin_y, xmax_y, ymin_y, ymax_y = (df_stretched.x.min(),
                                          df_stretched.x.max(),
                                          df_stretched.y.min(),
                                          df_stretched.y.max())

        #Some conditions unnecessary if it is assumed that electrodes don't overlap
        adjacent = df_corners[
            ((df_corners.x['min'] < xmax_x) & (df_corners.x['max'] >= xmax_x)
            # Check in x stretched direction
            |(df_corners.x['min'] < xmin_x) & (df_corners.x['max'] >= xmin_x))
            # Check if y is within bounds
            & (df_corners.y['min'] < ymax_x) & (df_corners.y['max'] > ymin_x) |

            #maybe do ymax_x - df_corners.y['min'] > threshold &
            #  df_corners.y['max'] - ymin_x > threshold

            ((df_corners.y['min'] < ymax_y) & (df_corners.y['max'] >= ymax_y)
             # Checks in y stretched direction
             |(df_corners.y['min'] < ymin_y) & (df_corners.y['max'] >= ymin_y))
             # Check if x in within bounds
            & ((df_corners.x['min'] < xmax_y) & (df_corners.x['max'] > xmin_y))
        ].index.values

        for shape in adjacent:
            temp_dict = {}
            reverse_dict = {}

            temp_dict ['source'] = shapeNumber
            reverse_dict['source'] = shape
            temp_dict ['target'] = shape
            reverse_dict['target'] = shapeNumber

            if(reverse_dict not in row_list):
                row_list.append(temp_dict)

    df_connected = (pd.DataFrame(row_list)[['source', 'target']]
                    .sort_index(axis=1, ascending=True)
                    .sort_values(['source', 'target']))
    return df_connected


Instruction:
Can you generate a brief explanation for the following Python 3 code
def extract_connections(svg_source, shapes_canvas, line_layer='Connections',
                        line_xpath=None, path_xpath=None, namespaces=None):
    '''
    Load all ``<svg:line>`` elements and ``<svg:path>`` elements from a layer
    of an SVG source.  For each element, if endpoints overlap distinct shapes
    in :data:`shapes_canvas`, add connection between overlapped shapes.

    .. versionchanged:: 0.6.post1
        Allow both ``<svg:line>`` *and* ``<svg:path>`` instances to denote
        connected/adjacent shapes.

    .. versionadded:: 0.6.post1
        :data:`path_xpath`

    Parameters
    ----------
    svg_source : filepath
        Input SVG file containing connection lines.
    shapes_canvas : shapes_canvas.ShapesCanvas
        Shapes canvas containing shapes to compare against connection
        endpoints.
    line_layer : str
        Name of layer in SVG containing connection lines.
    line_xpath : str
        XPath string to iterate through connection lines.
    path_xpath : str
        XPath string to iterate through connection paths.
    namespaces : dict
        SVG namespaces (compatible with :func:`etree.parse`).

    Returns
    -------
    pandas.DataFrame
        Each row corresponds to connection between two shapes in
        :data:`shapes_canvas`, denoted ``source`` and ``target``.
    '''
    from lxml import etree

    if namespaces is None:
        # Inkscape namespace is required to match SVG elements as well as
        # Inkscape-specific SVG tags and attributes (e.g., `inkscape:label`).
        namespaces = INKSCAPE_NSMAP

    # Parse SVG source.
    e_root = etree.parse(svg_source)
    # List to hold records of form: `[<id>, <x1>, <y1>, <x2>, <y2>]`.
    frames = []

    if line_xpath is None:
        # Define query to look for `svg:line` elements in top level of layer of
        # SVG specified to contain connections.
        line_xpath = ("//svg:g[@inkscape:label='%s']/svg:line" % line_layer)
    coords_columns = ['x1', 'y1', 'x2', 'y2']

    for line_i in e_root.xpath(line_xpath, namespaces=namespaces):
        # Extract start and end coordinate from `svg:line` element.
        line_i_dict = dict(list(line_i.items()))
        values = ([line_i_dict.get('id', None)] +
                  [float(line_i_dict[k]) for k in coords_columns])
        # Append record for end points of current line.
        frames.append(values)

    # Regular expression pattern to match start and end coordinates of
    # connection `svg:path` element.
    cre_path_ends = re.compile(r'^\s*M\s*(?P<start_x>\d+(\.\d+)?),\s*'
                               r'(?P<start_y>\d+(\.\d+)?).*'
                               # Diagonal line...
                               r'((L\s*(?P<end_x>\d+(\.\d+)?),\s*'
                               r'(?P<end_y>\d+(\.\d+)?))|'
                               # or Vertical line...
                               r'(V\s*(?P<end_vy>\d+(\.\d+)?))|'
                               # or Horizontal line
                               r'(H\s*(?P<end_hx>\d+(\.\d+)?))'
                               r')\D*'
                               r'$')

    if path_xpath is None:
        # Define query to look for `svg:path` elements in top level of layer of
        # SVG specified to contain connections.
        path_xpath = ("//svg:g[@inkscape:label='%s']/svg:path" % line_layer)

    for path_i in e_root.xpath(path_xpath, namespaces=namespaces):
        path_i_dict = dict(list(path_i.items()))
        match_i = cre_path_ends.match(path_i_dict['d'])
        if match_i:
            # Connection `svg:path` matched required format.  Extract start and
            # end coordinates.
            match_dict_i = match_i.groupdict()
            if match_dict_i['end_vy']:
                # Path ended with vertical line
                match_dict_i['end_x'] = match_dict_i['start_x']
                match_dict_i['end_y'] = match_dict_i['end_vy']
            if match_dict_i['end_hx']:
                # Path ended with horizontal line
                match_dict_i['end_x'] = match_dict_i['end_hx']
                match_dict_i['end_y'] = match_dict_i['start_y']
            # Append record for end points of current path.
            frames.append([path_i_dict['id']] + list(map(float,
                                                    (match_dict_i['start_x'],
                                                     match_dict_i['start_y'],
                                                     match_dict_i['end_x'],
                                                     match_dict_i['end_y']))))

    if not frames:
        return pd.DataFrame(None, columns=['source', 'target'])

    df_connection_lines = pd.DataFrame(frames, columns=['id'] + coords_columns)

    # Use `shapes_canvas.find_shape` to determine shapes overlapped by end
    # points of each `svg:path` or `svg:line`.
    df_shape_connections_i = pd.DataFrame([[shapes_canvas.find_shape(x1, y1),
                                            shapes_canvas.find_shape(x2, y2)]
                                           for i, (x1, y1, x2, y2) in
                                           df_connection_lines[coords_columns]
                                           .iterrows()],
                                          columns=['source', 'target'])
    # Order the source and target of each row so the source shape identifier is
    # always the lowest.
    df_shape_connections_i.sort_index(axis=1, inplace=True)
    # Tag each shape connection with the corresponding `svg:line`/`svg:path`
    # identifier.  May be useful, e.g., in debugging.
    df_shape_connections_i['line_id'] = df_connection_lines['id']
    # Remove connections where source or target shape was not matched (e.g., if
    # one or more end points does not overlap with a shape).
    return df_shape_connections_i.dropna()

Response: Extract all connection lines from a layer in a SVG file.


Instruction:
Explain what the following Python 3 code does
def discover():
    if CFG["plugins"]["autoload"]:
        report_plugins = CFG["plugins"]["reports"].value
        for plugin in report_plugins:
            try:
                importlib.import_module(plugin)
                LOG.debug("Found report: %s", plugin)
            except ImportError:
                LOG.error("Could not find '%s'", plugin)

Response:
Discover all experiments listed in benchbuild. reports. raw
AttributeNames


Instruction:
Can you tell what is the following Python 3 function doing
def index_name(self):
        fmt = self.campaign.export.elasticsearch.index_name
        fields = dict(date=self.report['date'])
        return fmt.format(**fields).lower()

Response: Get Elasticsearch index name associated to the campaign


Instruction:
Write a Python 3 script for
removing Elasticsearch index associated to the campaign

Response:
def remove_index(self):
        """Remove Elasticsearch index associated to the campaign"""
        self.index_client.close(self.index_name)
        self.index_client.delete(self.index_name)


Instruction:
How would you explain what the following Python 3 function does
def parse_args(args=None):
    from argparse import ArgumentParser

    if args is None:
        args = sys.argv

    parser = ArgumentParser(description='Rename template project with'
                            'hyphen-separated <new name> (path names and in '
                            'files).')
    parser.add_argument('new_name', help='New project name (e.g., '
                        ' `my-new-project`)')

    args = parser.parse_args()
    return args

Response: Parses arguments returns options and args.


Instruction:
Create a Python 3 function for
extracting the build information from a given executable.

Response:
def extract_build_info(exe_path, elf_section=ELF_SECTION):
    """Extracts the build information from a given executable.

    The build information is expected to be in json format, which is parsed
    and returned as a dictionary.
    If no build information is found an empty dictionary is returned.

    This assumes binutils 2.25 to work.

    Args:
        exe_path (str): The full path to the executable to be examined

    Returns:
        dict: A dictionary of the extracted information.
    """
    build_info = {}
    with mkdtemp() as tempd, pushd(tempd):
        proc = subprocess.Popen(
            [
                OBJCOPY,
                DUMP_SECTION,
                "{secn}={ofile}".format(secn=elf_section, ofile=BUILDINFO_FILE),
                exe_path,
            ],
            stderr=subprocess.PIPE,
        )
        proc.wait()
        errno = proc.returncode
        stderr = proc.stderr.read()
        if errno or len(stderr):  # just return the empty dict
            LOGGER.warning('objcopy failed with errno %s.', errno)
            if len(stderr):
                LOGGER.warning('objcopy failed with following msg:\n%s', stderr)
            return build_info

        with open(BUILDINFO_FILE) as build_info_f:
            try:
                build_info = json.load(build_info_f, object_hook=byteify)
            except JSONDcdError as jsde:
                LOGGER.warning('benchmark executable build is not valid json:')
                LOGGER.warning(jsde.msg)
                LOGGER.warning('build info section content:')
                LOGGER.warning(jsde.doc)
    return build_info


Instruction:
How would you code a function in Python 3 to
create a new entry in the cache.

Response:
def rk_create(self, im, im_type, timeout=60):
        """
        im: 图片字节
        im_type: 题目类型
        """
        params = {
            'typeid': im_type,
            'timeout': timeout,
        }
        params.update(self.base_params)
        files = {'image': ('check_code.png', im)}
        r = requests.post('http://api.ruokuai.com/create.json', data=params, files=files, headers=self.headers)
        return r.json()


Instruction:
Implement a Python 3 function for
reporting an error to the user.

Response:
def rk_report_error(self, im_id):
        """
        im_id:报错题目的ID
        """
        params = {
            'id': im_id,
        }
        params.update(self.base_params)
        r = requests.post('http://api.ruokuai.com/reporterror.json', data=params, headers=self.headers)
        return r.json()


Instruction:
Given the following Python 3 function, write the documentation
def lc_score(value):
    rebased = 2 * (value - 0.5)

    if rebased == 0:
        return 0
    elif rebased > 0:
        compliment = 1.0 - rebased
        score = - np.log2(compliment)
    else:
        compliment = 1.0 + rebased
        score = np.log2(compliment)
    return score

Response: Evaluates the accuracy of a predictive measure of a predictive measure.


Instruction:
How would you implement a function in Python 3 that
returns a customizable uchroot command.

Response:
def uchroot(*args, **kwargs):
    """
    Return a customizable uchroot command.

    Args:
        args: List of additional arguments for uchroot (typical: mounts)
    Return:
        chroot_cmd
    """
    uchroot_cmd = with_mounts(*args, uchroot_cmd_fn=no_llvm, **kwargs)
    return uchroot_cmd["--"]


Instruction:
Write a Python 3 script for
returning a customizable uchroot command.

Response:
def no_llvm(*args, uid=0, gid=0, **kwargs):
    """
    Return a customizable uchroot command.

    The command will be executed inside a uchroot environment.

    Args:
        args: List of additional arguments for uchroot (typical: mounts)
    Return:
        chroot_cmd
    """
    uchroot_cmd = no_args()
    uchroot_cmd = uchroot_cmd[__default_opts__(uid, gid)]
    return uchroot_cmd[args]


Instruction:
Write a Python 3 function for
returning the uchroot command without any customizations.

Response:
def no_args(**kwargs):
    """Return the uchroot command without any customizations."""
    from benchbuild.utils.cmd import uchroot as uchrt

    prefixes = CFG["container"]["prefixes"].value
    p_paths, p_libs = env(prefixes)
    uchrt = run.with_env_recursive(
        uchrt,
        LD_LIBRARY_PATH=path.list_to_path(p_libs),
        PATH=path.list_to_path(p_paths))

    return uchrt


Instruction:
Create a Python 3 function to
return a uchroot command with all mounts enabled.

Response:
def with_mounts(*args, uchroot_cmd_fn=no_args, **kwargs):
    """Return a uchroot command with all mounts enabled."""
    mounts = CFG["container"]["mounts"].value
    prefixes = CFG["container"]["prefixes"].value

    uchroot_opts, mounts = __mounts__("mnt", mounts)
    uchroot_cmd = uchroot_cmd_fn(**kwargs)
    uchroot_cmd = uchroot_cmd[uchroot_opts]
    uchroot_cmd = uchroot_cmd[args]
    paths, libs = env(mounts)
    prefix_paths, prefix_libs = env(prefixes)

    uchroot_cmd = run.with_env_recursive(
        uchroot_cmd,
        LD_LIBRARY_PATH=path.list_to_path(libs + prefix_libs),
        PATH=path.list_to_path(paths + prefix_paths))
    return uchroot_cmd


Instruction:
Can you create a Python 3 function that
returns a uchroot cmd that runs inside a filtered environment.

Response:
def clean_env(uchroot_cmd, varnames):
    """Returns a uchroot cmd that runs inside a filtered environment."""
    env = uchroot_cmd["/usr/bin/env"]
    __clean_env = env["-u", ",".join(varnames)]
    return __clean_env


Instruction:
Explain what the following Python 3 code does
def mounts(prefix, __mounts):
    i = 0
    mntpoints = []
    for mount in __mounts:
        if not isinstance(mount, dict):
            mntpoint = "{0}/{1}".format(prefix, str(i))
            mntpoints.append(mntpoint)
            i = i + 1
    return mntpoints

Response: Compute the mountpoints of the current user.


Instruction:
Create a Python 3 function to
compute the environment of the change root for the current user.

Response:
def env(mounts):
    """
    Compute the environment of the change root for the user.

    Args:
        mounts: The mountpoints of the current user.
    Return:
        paths
        ld_libs
    """
    f_mounts = [m.strip("/") for m in mounts]

    root = local.path("/")

    ld_libs = [root / m / "lib" for m in f_mounts]
    ld_libs.extend([root / m / "lib64" for m in f_mounts])

    paths = [root / m / "bin" for m in f_mounts]
    paths.extend([root / m / "sbin" for m in f_mounts])
    paths.extend([root / m for m in f_mounts])
    return paths, ld_libs


Instruction:
Can you tell what is the following Python 3 function doing
def get_abbreviations(kb):
    return {"%s$$n%i" % (author.get_urn(), i): abbrev
                for author in kb.get_authors()
                for i, abbrev in enumerate(author.get_abbreviations())
                if author.get_urn() is not None}

Response: Returns a list of all the abbreviations in the given kb.


Instruction:
Write a Python 3 function for
returning a dictionary like this :

Response:
def author_names(self):
        """
        Returns a dictionary like this:

        {
            "urn:cts:greekLit:tlg0012$$n1" : "Homer"
            , "urn:cts:greekLit:tlg0012$$n2" : "Omero"
            , ...
        }
        """
        return {"%s$$n%i" % (author.get_urn(), i): name[1]
                for author in self.get_authors()
                for i, name in enumerate(author.get_names())
                if author.get_urn() is not None}


Instruction:
Explain what the following Python 3 code does
def get_resource_by_urn(self, urn):
        search_query = """
            PREFIX frbroo: <http://erlangen-crm.org/efrbroo/>
            PREFIX crm: <http://erlangen-crm.org/current/>
            PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

            SELECT ?resource_URI

            WHERE {
                ?resource_URI crm:P1_is_identified_by ?urn .
                ?urn a crm:E42_Identifier .
                ?urn rdfs:label "%s"
            }
        """ % urn
        # check type of the input URN
        try:
            assert isinstance(urn, CTS_URN)
        except Exception as e:
            # convert to pyCTS.CTS_URN if it's a string
            urn = CTS_URN(urn)
            logger.debug('Converted the input urn from string to %s' % type(
                CTS_URN
            ))

        if (urn.work is not None):
            Work = self._session.get_class(surf.ns.EFRBROO['F1_Work'])
            result = self._store.execute_sparql(search_query)
            if len(result['results']['bindings']) == 0:
                raise ResourceNotFound
            else:
                tmp = result['results']['bindings'][0]
                resource_uri = tmp['resource_URI']['value']
                return self._session.get_resource(resource_uri, Work)

        elif (urn.work is None and urn.textgroup is not None):
            Person = self._session.get_class(surf.ns.EFRBROO['F10_Person'])
            result = self._store.execute_sparql(search_query)
            if len(result['results']['bindings']) == 0:
                raise ResourceNotFound
            else:
                tmp = result['results']['bindings'][0]
                resource_uri = tmp['resource_URI']['value']
                return self._session.get_resource(resource_uri, Person)

Response: Fetch the resource corresponding to the input URN.


Instruction:
Write a Python 3 function for
searching for a given string through the resources and returns an instance of HucitAuthor or HucitWork.

Response:
def search(self, search_string):
        """
        Searches for a given string through the resources' labels.

        :param search_string:
        :return: an instance of `HucitAuthor` or `HucitWork`.
        """
        query = """
        SELECT ?s ?label ?type
        WHERE {
            ?s a ?type .
            ?s rdfs:label ?label .
            ?label bif:contains "'%s'" .
        }
        """ % search_string
        response = self._session.default_store.execute_sparql(query)
        results = [(result['s']['value'], result['label']['value'], result['type']['value'])
                                                            for result in response["results"]["bindings"]]
        resources = [(label, self._session.get_resource(subject, self._session.get_class(type)))
                                                            for subject, label, type in results]

        Name = self._session.get_class(surf.ns.EFRBROO['F12_Name'])
        Title = self._session.get_class(surf.ns.EFRBROO['E35_Title'])
        Work = self._session.get_class(surf.ns.EFRBROO['F1_Work'])
        Person = self._session.get_class(surf.ns.EFRBROO['F10_Person'])

        result = []
        for label, resource in resources:
            if resource.uri == surf.ns.EFRBROO['E35_Title']:
                work = Work.get_by(efrbroo_P102_has_title = resource).first()
                result.append((label, work))
            elif resource.uri == surf.ns.EFRBROO['F12_Name']:
                author = Person.get_by(ecrm_P1_is_identified_by = resource).first()
                result.append((label, author))
            elif resource.uri == surf.ns.ECRM['E41_Appellation']:
                try:
                    name = Name.get_by(ecrm_P139_has_alternative_form = resource).first()
                    assert name is not None
                    author = Person.get_by(ecrm_P1_is_identified_by = name).first()
                    result.append((label, author))
                except Exception as e:
                    title = Title.get_by(ecrm_P139_has_alternative_form=resource).first()
                    assert title is not None
                    work = Work.get_by(efrbroo_P102_has_title = title).first()
                    result.append((label, work))
        return result


Instruction:
Write a Python 3 script to
return the authors in the Knowledge Base.

Response:
def get_authors(self):
        """
        Returns the authors in the Knowledge Base.

        :return: a list of `HucitAuthor` instances.

        """
        Person = self._session.get_class(surf.ns.EFRBROO['F10_Person'])
        return list(Person.all())


Instruction:
Write a Python 3 script for
returning the author s works.

Response:
def get_works(self):
        """Return the author's works.

        :return: a list of `HucitWork` instances.

        """
        Work = self._session.get_class(surf.ns.EFRBROO['F1_Work'])
        return list(Work.all())


Instruction:
Can you write a function in Python 3 where it
gets the label corresponding to the author identified by the CTS URN.

Response:
def get_author_label(self, urn):
        """Get the label corresponding to the author identified by the CTS URN.

        try to get an lang=en label (if multiple labels in this lang pick the shortest)
        try to get a lang=la label (if multiple labels in this lang exist pick the shortest)
        try to get a lang=None label (if multiple labels in this lang exist pick the shortest)

        returns None if no name is found

        """
        author = self.get_resource_by_urn(urn)
        names = author.get_names()
        en_names = sorted([name[1] for name in names if name[0] == "en"], key=len)
        try:
            assert len(en_names) > 0
            return en_names[0]
        except Exception as e:
            none_names = sorted([name[1] for name in names if name[0] == None], key=len)
            try:
                return none_names[0]
            except Exception as e:
                la_names = sorted([name[1] for name in names if name[0] == "la"], key=len)
                try:
                    assert len(la_names) > 0
                    return la_names[0]
                except Exception as e:
                    return None


Instruction:
Implement a function in Python 3 to
gather basic stats about the Knowledge Base and its contents.

Response:
def get_statistics(self):
        """
        Gather basic stats about the Knowledge Base and its contents.

        :return: a dictionary

        """
        statistics = {
            "number_authors": 0,
            "number_author_names": 0,
            "number_author_abbreviations": 0,
            "number_works": 0,
            "number_work_titles": 0,
            "number_title_abbreviations": 0,
            "number_opus_maximum":0,
        }
        for author in self.get_authors():
            if author.get_urn() is not None:
                opmax = True if self.get_opus_maximum_of(author.get_urn())\
                    is not None else False
                if opmax:
                    statistics["number_opus_maximum"] += 1
            statistics["number_authors"] += 1
            statistics["number_author_names"] += len(author.get_names())
            statistics["number_author_abbreviations"] += len(
                author.get_abbreviations()
            )
            for work in author.get_works():
                statistics["number_works"] += 1
                statistics["number_work_titles"] += len(work.get_titles())
                statistics["number_title_abbreviations"] += len(
                    work.get_abbreviations()
                )
        return statistics


Instruction:
Explain what the following Python 3 code does
def get_opus_maximum_of(self, author_cts_urn):
        author = self.get_resource_by_urn(author_cts_urn)
        assert author is not None
        works = author.get_works()

        if len(works) > 1:
            for work in works:
                if work.is_opus_maximum():
                    return work
        elif len(works) == 1:
            return works[0]
        else:
            return None

Response: Returns the opus maximum of an author.


Instruction:
Can you generate a brief explanation for the following Python 3 code
def to_json(self):
        return json.dumps({
            "statistics": self.get_statistics()
            , "authors": [json.loads(author.to_json()) for author in self.get_authors()]
        }, indent=2)

Response: Serialises the content of the KnowledgeBase as JSON.


Instruction:
Here you have a function in Python 3, explain what it does
def write_yaml_report(func):

    @wraps(func)
    def _wrapper(*args, **kwargs):
        now = datetime.datetime.now()
        with Timer() as timer:
            data = func(*args, **kwargs)
            if isinstance(data, (SEQUENCES, types.GeneratorType)):
                report = dict(children=list(map(str, data)))
            elif isinstance(data, MAPPINGS):
                report = data
            else:
                raise Exception('Unexpected data type: %s', type(data))
        report['elapsed'] = timer.elapsed
        report['date'] = now.isoformat()
        if "no_exec" not in kwargs and report is not None:
            with open(YAML_REPORT_FILE, 'w') as ostr:
                yaml.dump(report, ostr, default_flow_style=False)
        return report

    return _wrapper

Response: Decorator used in campaign node post - processing


Instruction:
Can you create a Python 3 function that
classes function that MUST be specified as decorator to the `__call__` method overriden by sub-classes.

Response:
def call_decorator(cls, func):
        """class function that MUST be specified as decorator
        to the `__call__` method overriden by sub-classes.
        """

        @wraps(func)
        def _wrap(self, *args, **kwargs):
            try:
                return func(self, *args, **kwargs)
            except Exception:
                self.logger.exception('While executing benchmark')
                if not (self.catch_child_exception or False):
                    raise

        return _wrap


Instruction:
Can you implement a function in Python 3 that
enumerates children and build associated objects

Response:
def traverse(self):
        """Enumerate children and build associated objects
        """
        builder = self.child_builder
        for child in self._children:
            with pushd(str(child)):
                yield child, builder(child)


Instruction:
Here you have a function in Python 3, explain what it does
def i_ww(self):
        d_values = []
        for i in range(self.n_pads_l):
            d_values.append(self.pad_position_l(i))
        d_values = np.array(d_values) - self.length / 2
        area_d_sqrd = sum(self.pad_area * d_values ** 2) * self.n_pads_w
        i_second = self.pad_i_ww * self.n_pads
        return area_d_sqrd + i_second

Response: Second moment of inertia around the width axis.


Instruction:
How would you explain what the following Python 3 function does
def i_ll(self):
        d_values = []
        for i in range(self.n_pads_w):
            d_values.append(self.pad_position_w(i))
        d_values = np.array(d_values) - self.width / 2
        area_d_sqrd = sum(self.pad_area * d_values ** 2) * self.n_pads_l
        i_second = self.pad_i_ll * self.n_pads
        return area_d_sqrd + i_second

Response: Second moment of inertia around the length axis.


Instruction:
Can you create a Python 3 function that
determines the position of the ith pad in the length direction.

Response:
def pad_position_l(self, i):
        """
        Determines the position of the ith pad in the length direction.
        Assumes equally spaced pads.

        :param i: ith number of pad in length direction (0-indexed)
        :return:
        """
        if i >= self.n_pads_l:
            raise ModelError("pad index out-of-bounds")
        return (self.length - self.pad_length) / (self.n_pads_l - 1) * i + self.pad_length / 2


Instruction:
Write a Python 3 function for
determining the position of the ith pad in the width direction.

Response:
def pad_position_w(self, i):
        """
        Determines the position of the ith pad in the width direction.
        Assumes equally spaced pads.

        :param i: ith number of pad in width direction (0-indexed)
        :return:
        """
        if i >= self.n_pads_w:
            raise ModelError("pad index out-of-bounds")
        return (self.width - self.pad_width) / (self.n_pads_w - 1) * i + self.pad_width / 2


Instruction:
Create a Python 3 function for
adding paths to the given batch object.

Response:
def add_to_batch(self, batch):
        '''
        Adds paths to the given batch object. They are all added as
        GL_TRIANGLES, so the batch will aggregate them all into a single OpenGL
        primitive.
        '''
        for name in self.paths:
            svg_path = self.paths[name]
            svg_path.add_to_batch(batch)


Instruction:
Implement a function in Python 3 to
parse all path elements from xml_root and return a dict of the attributes.

Response:
def parse(self, xml_root, on_error=None):
        '''
        Parse all <path> elements from xml_root.

        Optional on_error arg specifies a callback function to be run when
        an error occurs during parsing.
        The specified on_error function must accept 3 arguments:
            <svg filename>, <path_tag>, <error message>
        An example on_error handler is provided as svg_load.svg_parser.parse_warning(),
        where all SvgParseErrors are converted to warning messages.  See usage below:
        >>> import re
        >>> svg_parser = SvgParser()
        >>> path_tag = etree.XML("""
        ...     <path
        ...     xmlns="http://www.w3.org/2000/svg"
        ...     xmlns:dc="http://purl.org/dc/elements/1.1/"
        ...     xmlns:cc="http://creativecommons.org/ns#"
        ...     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
        ...     xmlns:svg="http://www.w3.org/2000/svg"
        ...     xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
        ...     xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
        ...     id="path13231"
        ...     d="M8 4 l-4,4"
        ...     linecap="square"
        ...     stroke="#000000"
        ...     stroke-width="0.25"
        ...     />""")
        >>> with warnings.catch_warnings(record=True) as w:
        ...     svg = svg_parser.parse(path_tag, on_error=parse_warning)
        >>> print w[-1].category
        <type 'exceptions.RuntimeWarning'>
        >>> match = re.search(r'^Error parsing None:\d+, unsupported svg path command: l', str(w[-1].message))
        >>> print match is None
        False
        >>> path_tag = etree.XML("""
        ...     <path
        ...     xmlns="http://www.w3.org/2000/svg" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
        ...     xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#"
        ...     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
        ...     xmlns:svg="http://www.w3.org/2000/svg"
        ...     xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
        ...     style="fill:#0000ff;stroke:#ff0000;stroke-width:0.10000000000000001;stroke-miterlimit:4;stroke-dasharray:none"
        ...     id="path18327"
        ...     d="M 636.0331,256.9345 L 636.0331,256.9345"
        ...     inkscape:connector-curvature="0"/>""")
        >>> with warnings.catch_warnings(record=True) as w:
        ...     svg = svg_parser.parse(path_tag, on_error=parse_warning)
        >>> print w[-1].category
        <type 'exceptions.RuntimeWarning'>
        >>> match = re.search(r'^Error parsing None:\d+, loop needs 3 or more verts', str(w[-1].message))
        >>> print match is None
        False
        '''
        svg = Svg()
        svg_namespace = {'svg': 'http://www.w3.org/2000/svg'}
        path_tags = xml_root.xpath('(/svg:svg|/svg:svg/svg:g)/svg:path',
                                   namespaces=svg_namespace)
        parser = PathParser()
        for path_tag in path_tags:
            try:
                id, svg_path = parser.parse(path_tag)
                if svg_path.loops:
                    svg.add_path(id, svg_path)
            except (ParseError, ) as why:
                filename = getattr(self, 'filename', None)
                args = (filename, path_tag, why.message)
                if on_error:
                    on_error(*args)
                else:
                    raise SvgParseError(*args)

        if svg.paths:
            x, y = svg.get_boundary().get_center()
            for svg_path in svg.paths.values():
                svg_path.offset(-x, -y)
        return svg


Instruction:
Write a Python 3 script for
turning a string into a list of string and number chunks.

Response:
def alphanum_key(s):
    """Turn a string into a list of string and number chunks.

        "z23a" -> ["z", 23, "a"]
        """

    return [int(c) if c.isdigit() else c for c in _RE_INT.split(s)]


Instruction:
Write a Python 3 script for
discovering all experiments listed in PLUGINS_EXPERIMENTS.

Response:
def discover():
    """
    Import all experiments listed in PLUGINS_EXPERIMENTS.

    Tests:
        >>> from benchbuild.settings import CFG
        >>> from benchbuild.experiments import discover
        >>> import logging as lg
        >>> import sys
        >>> l = lg.getLogger('benchbuild')
        >>> lg.getLogger('benchbuild').setLevel(lg.DEBUG)
        >>> lg.getLogger('benchbuild').handlers = [lg.StreamHandler(stream=sys.stdout)]
        >>> CFG["plugins"]["experiments"] = ["benchbuild.non.existing", "benchbuild.experiments.raw"]
        >>> discover()
        Could not find 'benchbuild.non.existing'
        ImportError: No module named 'benchbuild.non'
    """
    if CFG["plugins"]["autoload"]:
        experiment_plugins = CFG["plugins"]["experiments"].value
        for exp_plugin in experiment_plugins:
            try:
                importlib.import_module(exp_plugin)
            except ImportError as import_error:
                LOG.error("Could not find '%s'", exp_plugin)
                LOG.error("ImportError: %s", import_error.msg)