Question:
How to multiply two 2D RFFT arrays (FFTPACK) to be compatible with NumPy's FFT? I'm trying to multiply two 2D arrays that were transformed with fftpack_rfft2d() (SciPy's FFTPACK RFFT) and the result is not compatible with what I get from scipy_rfft2d() (SciPy's FFT RFFT). The image below shares the output of the script, which displays: The initialization values of both input arrays; Both arrays after they were transform with SciPy's FFT implementation for RFFT using scipy_rfft2d(), followed by the output of the multiplication after its transformed backwards with scipy_irfft2d(); The same things using SciPy's FFTPACK implementation for RFFT with fftpack_rfft2d() and fftpack_irfft2d(); The result of a test with np.allclose() that checks if the result of both multiplications are the same after they were transformed back with their respective implementations for IRFFT. Just to be clear, the red rectangles display the multiplication result after the inverse transform IRFFT: the rectangle on the left uses SciPy's FFT IRFFT; the rectangle on the right, SciPy's FFTPACK IRFFT. They should present the same data when the multiplication with the FFTPACK version is fixed. I think the multiplication result with the FFTPACK version is not correct because scipy.fftpack returns the real and imaginary parts in the resulting RFFT array differently than the RFFT from scipy.fft: I believe that RFFT from scipy.fftpack returns an array where one element contains the real part and the next element holds its imaginary counterpart; In RFFT from scipy.fft, each element is a complex number and therefore is able to hold the real and imaginary parts simultaneously; Please correct me if I'm wrong! I would also like to point out that since scipy.fftpack doesn't provide functions for transforming 2D arrays like rfft2() and irfft2(), I'm providing my own implementations in the code below:
```
import numpy as np
from scipy import fftpack as scipy_fftpack
from scipy import fft as scipy_fft

# SCIPY RFFT 2D
def scipy_rfft2d(matrix):
    fftRows = [scipy_fft.rfft(row) for row in matrix]
    return np.transpose([scipy_fft.fft(row) for row in np.transpose(fftRows)])

# SCIPY IRFFT 2D
def scipy_irfft2d(matrix, s):
    fftRows = [scipy_fft.irfft(row) for row in matrix]
    return np.transpose([scipy_fft.ifft(row) for row in np.transpose(fftRows)])

# FFTPACK RFFT 2D
def fftpack_rfft2d(matrix):
    fftRows = [scipy_fftpack.rfft(row) for row in matrix]
    return np.transpose([scipy_fftpack.rfft(row) for row in np.transpose(fftRows)])

# FFTPACK IRFFT 2D
def fftpack_irfft2d(matrix):
    fftRows = [scipy_fftpack.irfft(row) for row in matrix]
    return np.transpose([scipy_fftpack.irfft(row) for row in np.transpose(fftRows)])

print('\n####################     INPUT DATA     ###################\n')

# initialize two 2D arrays with random data for testing
in1 = np.array([[0,   0,   0,   0], \
                [0, 255, 255,   0], \
                [0,   0, 255, 255], \
                [0,   0,   0,   0]])

print('\nin1 shape=', in1.shape, '\n', in1)

in2 = np.array([[0,   0,   0,   0], \
                [0,   0, 255,   0], \
                [0, 255, 255,   0], \
                [0, 255,   0,   0]])

print('\nin2 shape=', in2.shape, '\n', in2)

print('\n###############    SCIPY: 2D RFFT (MULT)    ###############\n')

# transform both inputs with SciPy RFFT for 2D
scipy_rfft1 = scipy_fft.rfftn(in1)
scipy_rfft2 = scipy_fft.rfftn(in2)

print('* Output from scipy_fft.rfftn():')
print('scipy_fft1 shape=', scipy_rfft1.shape, '\n', scipy_rfft1.real)
print('\nscipy_fft2 shape=', scipy_rfft2.shape, '\n', scipy_rfft2.real)

# perform multiplication between two 2D arrays from SciPy RFFT
scipy_rfft_mult = scipy_rfft1 * scipy_rfft2

# perform inverse RFFT for 2D arrays using SciPy
scipy_data = scipy_fft.irfftn(scipy_rfft_mult, in1.shape) # passing shape guarantees the output will have the original data size
print('\n* Output from scipy_fft.irfftn():')
print('scipy_data shape=', scipy_data.shape, '\n', scipy_data)

print('\n###############   FFTPACK: 2D RFFT (MULT)   ###############\n')

# transform both inputs with FFTPACK RFFT for 2D
fftpack_rfft1 = fftpack_rfft2d(in1)
fftpack_rfft2 = fftpack_rfft2d(in2)
print('* Output from fftpack_rfft2d():')
print('fftpack_rfft1 shape=', fftpack_rfft1.shape, '\n', fftpack_rfft1)
print('\nfftpack_rfft2 shape=', fftpack_rfft2.shape, '\n', fftpack_rfft2)

# TODO: perform multiplication between two 2D arrays from FFTPACK RFFT
fftpack_rfft_mult = fftpack_rfft1 * fftpack_rfft2 # this doesn't work

# perform inverse RFFT for 2D arrays using FFTPACK
fftpack_data = fftpack_irfft2d(fftpack_rfft_mult)
print('\n* Output from fftpack_irfft2d():')
print('fftpack_data shape=', fftpack_data.shape, '\n', fftpack_data)

print('\n#####################      RESULT     #####################\n')

# compare FFTPACK result with SCIPY
print('\nIs fftpack_data equivalent to scipy_data?', np.allclose(fftpack_data, scipy_data), '\n')
```
Assuming my guess is correct, what would be the correct implementation for a function that multiplies two 2D arrays that were generated from fftpack_rfft2d()? Remember: the resulting array must be able to be transformed back with fftpack_irfft2d(). Only answers that address the problem in 2-dimensions are invited. Those interested in how to multiply 1D FFTPACK arrays can check this thread.

Response:
Correct functions:
```
import numpy as np
from scipy import fftpack as scipy_fftpack
from scipy import fft as scipy

# FFTPACK RFFT 2D
def fftpack_rfft2d(matrix):
    fftRows = scipy_fftpack.fft(matrix, axis=1)
    fftCols = scipy_fftpack.fft(fftRows, axis=0)

    return fftCols

# FFTPACK IRFFT 2D
def fftpack_irfft2d(matrix):
    ifftRows = scipy_fftpack.ifft(matrix, axis=1)
    ifftCols = scipy_fftpack.ifft(ifftRows, axis=0)

    return ifftCols.real
```
You calculated the 2D FFT in wrong way. Yes, the first FFT (by columns in your case) can be calculated using rfft(), but the second FFT calculation must be provided on the complex output of the first FFT (by columns), so the output of the rfft() must be converted into true complex spectrum. Moreover, this mean, that you must use fft() instead of rfft() for the second FFT by rows. Consiquently, it is more convenient to use fft() in both calculations. Moreover, you have input data as a numpy 2D arrays, why do you use list comprehension? Use fftpack.fft() directly, this is much faster. If you already have only 2D arrays calculated by wrong functions and need multiply them: then, my opinion, to try reconstruct the input data from the wrong 2D FFT using the same 'wrong' way and then calculate correct 2D FFT ================================================================ The full testing code with new functions version:
```
import numpy as np
from scipy import fftpack as scipy_fftpack
from scipy import fft as scipy_fft

# FFTPACK RFFT 2D
def fftpack_rfft2d(matrix):
    fftRows = scipy_fftpack.fft(matrix, axis=1)
    fftCols = scipy_fftpack.fft(fftRows, axis=0)

    return fftCols

# FFTPACK IRFFT 2D
def fftpack_irfft2d(matrix):
    ifftRows = scipy_fftpack.ifft(matrix, axis=1)
    ifftCols = scipy_fftpack.ifft(ifftRows, axis=0)

    return ifftCols.real

print('\n####################     INPUT DATA     ###################\n')

# initialize two 2D arrays with random data for testing
in1 = np.array([[0,   0,   0,   0], \
                [0, 255, 255,   0], \
                [0,   0, 255, 255], \
                [0,   0,   0,   0]])

print('\nin1 shape=', in1.shape, '\n', in1)

in2 = np.array([[0,   0,   0,   0], \
                [0,   0, 255,   0], \
                [0, 255, 255,   0], \
                [0, 255,   0,   0]])

print('\nin2 shape=', in2.shape, '\n', in2)

print('\n###############    SCIPY: 2D RFFT (MULT)    ###############\n')

# transform both inputs with SciPy RFFT for 2D
scipy_rfft1 = scipy_fft.fftn(in1)
scipy_rfft2 = scipy_fft.fftn(in2)

print('* Output from scipy_fft.rfftn():')
print('scipy_fft1 shape=', scipy_rfft1.shape, '\n', scipy_rfft1)
print('\nscipy_fft2 shape=', scipy_rfft2.shape, '\n', scipy_rfft2)

# perform multiplication between two 2D arrays from SciPy RFFT
scipy_rfft_mult = scipy_rfft1 * scipy_rfft2

# perform inverse RFFT for 2D arrays using SciPy
scipy_data = scipy_fft.irfftn(scipy_rfft_mult, in1.shape) # passing shape guarantees the output will
                                                          # have the original data size
print('\n* Output from scipy_fft.irfftn():')
print('scipy_data shape=', scipy_data.shape, '\n', scipy_data)

print('\n###############   FFTPACK: 2D RFFT (MULT)   ###############\n')

# transform both inputs with FFTPACK RFFT for 2D
fftpack_rfft1 = fftpack_rfft2d(in1)
fftpack_rfft2 = fftpack_rfft2d(in2)
print('* Output from fftpack_rfft2d():')
print('fftpack_rfft1 shape=', fftpack_rfft1.shape, '\n', fftpack_rfft1)
print('\nfftpack_rfft2 shape=', fftpack_rfft2.shape, '\n', fftpack_rfft2)

# TODO: perform multiplication between two 2D arrays from FFTPACK RFFT
fftpack_rfft_mult = fftpack_rfft1 * fftpack_rfft2 # this doesn't work

# perform inverse RFFT for 2D arrays using FFTPACK
fftpack_data = fftpack_irfft2d(fftpack_rfft_mult)
print('\n* Output from fftpack_irfft2d():')
print('fftpack_data shape=', fftpack_data.shape, '\n', fftpack_data)

print('\n#####################      RESULT     #####################\n')

# compare FFTPACK result with SCIPY
print('\nIs fftpack_data equivalent to scipy_data?', np.allclose(fftpack_data, scipy_data), '\n')
```
The output is:
```
####################     INPUT DATA     ###################

in1 shape= (4, 4)
 [[  0   0   0   0]
 [  0 255 255   0]
 [  0   0 255 255]
 [  0   0   0   0]]

in2 shape= (4, 4)
 [[  0   0   0   0]
 [  0   0 255   0]
 [  0 255 255   0]
 [  0 255   0   0]]

###############    SCIPY: 2D RFFT (MULT)    ###############

* Output from scipy_fft.rfftn():
scipy_fft1 shape= (4, 4)
 [[1020.  -0.j -510.  +0.j    0.  -0.j -510.  -0.j]
 [-510.-510.j    0.  +0.j    0.  +0.j  510.+510.j]
 [   0.  -0.j    0.+510.j    0.  -0.j    0.-510.j]
 [-510.+510.j  510.-510.j    0.  -0.j    0.  -0.j]]

scipy_fft2 shape= (4, 4)
 [[1020.  -0.j -510.-510.j    0.  -0.j -510.+510.j]
 [-510.  +0.j  510.+510.j    0.-510.j    0.  -0.j]
 [   0.  -0.j    0.  +0.j    0.  -0.j    0.  -0.j]
 [-510.  -0.j    0.  +0.j    0.+510.j  510.-510.j]]

* Output from scipy_fft.irfftn():
scipy_data shape= (4, 4)
 [[130050.  65025.  65025. 130050.]
 [ 65025.      0.      0.  65025.]
 [ 65025.      0.      0.  65025.]
 [130050.  65025.  65025. 130050.]]

###############   FFTPACK: 2D RFFT (MULT)   ###############

* Output from fftpack_rfft2d():
fftpack_rfft1 shape= (4, 4)
 [[1020.  -0.j -510.  +0.j    0.  -0.j -510.  +0.j]
 [-510.-510.j    0.  +0.j    0.  +0.j  510.+510.j]
 [   0.  +0.j    0.+510.j    0.  +0.j    0.-510.j]
 [-510.+510.j  510.-510.j    0.  +0.j    0.  +0.j]]

fftpack_rfft2 shape= (4, 4)
 [[1020.  -0.j -510.-510.j    0.  -0.j -510.+510.j]
 [-510.  +0.j  510.+510.j    0.-510.j    0.  +0.j]
 [   0.  +0.j    0.  +0.j    0.  +0.j    0.  +0.j]
 [-510.  +0.j    0.  +0.j    0.+510.j  510.-510.j]]

* Output from fftpack_irfft2d():
fftpack_data shape= (4, 4)
 [[130050.+0.j  65025.+0.j  65025.+0.j 130050.+0.j]
 [ 65025.+0.j      0.+0.j      0.+0.j  65025.+0.j]
 [ 65025.+0.j      0.+0.j      0.+0.j  65025.+0.j]
 [130050.+0.j  65025.+0.j  65025.-0.j 130050.+0.j]]

#####################      RESULT     #####################

Is fftpack_data equivalent to scipy_data? True
```


Question:
How do I access the elements of a slice object in python I'm using scipy.ndimage.label and scipy.ndimage.find_objects to find things in an image. It returns an array of slices. I am trying to get the coordinates of the object out of the slice but can't seem to find a way to get inside the slice object. Here is a simple example:
```
a = 1
b = 2
c = 13
d = 47
j = slice(a,b,None)
k = slice(c,d,None)
x = (j, k)

print(x)
print(x[0])
print(x[0].indices(2))
print(x[1].indices(2))
```
Output is:
```
(slice(1, 2, None), slice(13, 47, None))
slice(1, 2, None)
(1, 2, 1)
(2, 2, 1)
```
Basically I am looking for the ability to get the values for a, b, c, and d if I'm only given the slice tuple x. I thought indices would get me on the way but I'm not understanding it's behavior.

Response:
Are you looking for the start, stop and step properties?
```

```python
s = slice(1, 2, 3)
```

```python
s.start
#Output
#1
```

```python
s.stop
#Output
#2
```

```python
s.step
#Output
#3
#```
#slice.indices computes the start/stop/step for the indices that would be accessed for an iterable with the input length. So,
#```
```

```python
s = slice(-1, None, None)
```

```python
s.indices(30)
#Output
#(29, 30, 1)
#```
#Which means that you would take item 29 from the iterable. It is able to be conveniently combined with xrange (or range):
#```
#for item in range(*some_slice.indices(len(sequence))):
#    print(sequence[item])
#```
#As a concrete example:
#```
```

```python
a = range(30)
```

```python
for i in a[-2:]:
  print(i)

#Output
#28
#29
```

```python
s = slice(-2, None, None)
```

```python
for ix in range(*s.indices(len(a))):
  print(a[ix])

#Output
#28
#29
#```
```


Question:
How to test for uniformity I simulate times in the range 0 to T according to a Poisson process. The inter-event times are exponential and we know that the distribution of the times should be uniform in the range 0 to T.
```
def poisson_simul(rate, T):
    time = random.expovariate(rate)
    times = [0]
    while (times[-1] < T):
        times.append(time+times[-1])
        time = random.expovariate(rate)
    return times[1:]
```
I would simply like to run one of the tests for uniformity, for example the Kolmogorov-Smirnov test. I can't work out how to do this in scipy however. If I do
```
import random
from scipy.stats import kstest
times = poisson_simul(1, 100)
print kstest(times, "uniform")
```
it is not right . It gives me
```
(1.0, 0.0)
```
I just want to test the hypothesis that the points are uniformly chosen from the range 0 to T. How do you do this in scipy?

Response:
You need to provide the parameters of the uniform distribution to let kstest() know that it is a uniform distribution from 0 to 100. If you just specify 'uniform', you get the default bounds of 0 to 1, which the data obviously does not fit. The clearest way to do this is to specify the CDF function directly instead of using the strings:
```
[~]
|11> from scipy import stats

[~]
|12> times = poisson_simul(1.0, 100.0)

[~]
|13> stats.kstest(times, stats.uniform(loc=0.0, scale=100.0).cdf)
(0.047464592615975507, 0.98954417186125665)
```


Question: How do you compute the confidence interval for Pearson's r in Python? In Python, I know how to calculate r and associated p-value using scipy.stats.pearsonr, but I'm unable to find a way to calculate the confidence interval of r. How is this done? Thanks for any help :)

Response:
According to [1], calculation of confidence interval directly with Pearson r is complicated due to the fact that it is not normally distributed. The following steps are needed: Convert r to z', Calculate the z' confidence interval. The sampling distribution of z' is approximately normally distributed and has standard error of 1/sqrt(n-3). Convert the confidence interval back to r. Here are some sample codes:
```
def r_to_z(r):
    return math.log((1 + r) / (1 - r)) / 2.0

def z_to_r(z):
    e = math.exp(2 * z)
    return((e - 1) / (e + 1))

def r_confidence_interval(r, alpha, n):
    z = r_to_z(r)
    se = 1.0 / math.sqrt(n - 3)
    z_crit = stats.norm.ppf(1 - alpha/2)  # 2-tailed z critical value

    lo = z - z_crit * se
    hi = z + z_crit * se

    # Return a sequence
    return (z_to_r(lo), z_to_r(hi))
```
Reference: http://onlinestatbook.com/2/estimation/correlation_ci.html


Question:
2D Nearest Neighbor Interpolation in Python Suppose that we have the following look up table
```
| 1.23    2.63    4.74    6.43    5.64
 -------|--------------------------------------
 -------|--------------------------------------
 2.56   |  0       0      1        0       1
 4.79   |  0       1      1        1       0
 6.21   |  1       0      0        0       0
```
This table contains a labeling matrix (having only 0 and 1s), x values and y values. How one can have nearest-neighbor interpolation for this look up table? Example:
```
Input: (5.1, 4.9)
Output: 1

Input: (3.54, 6.9)
Output: 0
```

Response:
Look up table If you have the complete table you don't need interpolation, you just need to look up the index of the nearest (x, y) value and use it on the table
```

```python
import numpy
   ...: x = numpy.array([1.23, 2.63, 4.74, 6.43, 5.64])
   ...: y = numpy.array([2.56, 4.79, 6.21])
   ...: data = numpy.array([[0, 0, 1, 0, 1],
   ...:                     [0, 1, 1, 1, 0],
   ...:                     [1, 0, 0, 0, 0]])
   ...:
   ...: def lookupNearest(x0, y0):
   ...:     xi = numpy.abs(x-x0).argmin()
   ...:     yi = numpy.abs(y-y0).argmin()
   ...:     return data[yi,xi]
```

```python
lookupNearest(5.1, 4.9)
#Output
#1
```

```python
lookupNearest(3.54, 6.9)
#Output
#0
#```
#Nearest-neighbor interpolation scipy.interpolate.NearestNDInterpolator will be really useful if your data is composed by scattered points For example, for data like: with red = 1, blue =0
#```
```

```python
points = numpy.array([[1.1, 2.5],
   ...:                       [1.5, 5.2],
   ...:                       [3.1, 3.0],
   ...:                       [2.0, 6.0],
   ...:                       [2.8, 4.7]])
   ...: values = numpy.array([0, 1, 1, 0, 0])
```

```python
from scipy.interpolate import NearestNDInterpolator
   ...: myInterpolator = NearestNDInterpolator(points, values)
```

```python
myInterpolator(1.7,4.5)
#Output
#1
```

```python
myInterpolator(2.5,4.0)
#Output
#0
#```
```


Question:
Scipy interp1d and matlab interp1 The following are the inputs for my interpolation:
```
x = [-1.01, 5.66, 5.69, 13.77, 20.89]

y = [0.28773, 1.036889, 1.043178, 1.595322, 1.543763]

new_x = [0, 2, 4, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20]
```
The results from matlab interp1 and scipy.interpolate interp1d are different. The results are like this.
```
new_y_scipy=[0.401171, 0.625806, 0.850442, 1.062384, 1.186291, 1.248244, 1.310198, 1.372152, 1.434105, 1.496059, 1.545429, 1.55267, 1.559911, 1.567153, 1.574394, 1.588877,]

new_y_matlab=[0.401171, 0.625806, 0.850442, 1.064362, 1.201031, 1.269366, 1.3377, 1.406035, 1.47437, 1.542704, 1.593656, 1.586415, 1.579174, 1.571932, 1.564691, 1.550208]
```
Apparently matlab seems to get better result than scipy. What is the fundamental difference?

Response:
I think that your data from scipy might be messed up somehow, because I can't reproduce your problem. For me, the results from scipy match perfectly with your results from matlab. See below for a demonstration:
```
import numpy as np
from scipy.interpolate import interp1d
import matplotlib.pyplot as plt

x = [-1.01, 5.66, 5.69, 13.77, 20.89]
y = [0.28773, 1.036889, 1.043178, 1.595322, 1.543763]

new_x = [0, 2, 4, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20]
new_y_scipy=[0.401171, 0.625806, 0.850442, 1.062384, 1.186291, 1.248244, 1.310198, 1.372152, 1.434105, 1.496059, 1.545429, 1.55267, 1.559911, 1.567153, 1.574394, 1.588877,]
new_y_matlab=[0.401171, 0.625806, 0.850442, 1.064362, 1.201031, 1.269366, 1.3377, 1.406035, 1.47437, 1.542704, 1.593656, 1.586415, 1.579174, 1.571932, 1.564691, 1.550208]

askewchan = interp1d(x,y)(new_x)

# 'linear' has no effect since it's the default, but I'll plot it too:
set_interp = interp1d(x, y, kind='linear')
new_y = set_interp(new_x)

plt.plot(x, y, 'o', new_x, new_y_scipy, '--', new_x, new_y_matlab, ':', new_x, askewchan, '.', new_x, new_y, '+')
plt.legend(('Original','OP_scipy', 'OP_matlab', 'askewchan_scipy', 'OP style scipy'), loc='lower right')

np.allclose(new_y_matlab, interp1d(x,y)(new_x))
#True
```


Question:
How to speed-up nested loop? I'm performing a nested loop in python that is included below. This serves as a basic way of searching through existing financial time series and looking for periods in the time series that match certain characteristics. In this case there are two separate, equally sized, arrays representing the 'close' (i.e. the price of an asset) and the 'volume' (i.e. the amount of the asset that was exchanged over the period). For each period in time I would like to look forward at all future intervals with lengths between 1 and INTERVAL_LENGTH and see if any of those intervals have characteristics that match my search (in this case the ratio of the close values is greater than 1.0001 and less than 1.5 and the summed volume is greater than 100). My understanding is that one of the major reasons for the speedup when using NumPy is that the interpreter doesn't need to type-check the operands each time it evaluates something so long as you're operating on the array as a whole (e.g. numpy_array * 2), but obviously the code below is not taking advantage of that. Is there a way to replace the internal loop with some kind of window function which could result in a speedup, or any other way using numpy/scipy to speed this up substantially in native python? Alternatively, is there a better way to do this in general (e.g. will it be much faster to write this loop in C++ and use weave)?
```
ARRAY_LENGTH = 500000
INTERVAL_LENGTH = 15
close = np.array( xrange(ARRAY_LENGTH) )
volume = np.array( xrange(ARRAY_LENGTH) )
close, volume = close.astype('float64'), volume.astype('float64')

results = []
for i in xrange(len(close) - INTERVAL_LENGTH):
    for j in xrange(i+1, i+INTERVAL_LENGTH):
        ret = close[j] / close[i]
        vol = sum( volume[i+1:j+1] )
        if ret > 1.0001 and ret < 1.5 and vol > 100:
            results.append( [i, j, ret, vol] )
print results
```

Response:
Update: (almost) completely vectorized version below in "new_function2"... I'll add comments to explain things in a bit. It gives a ~50x speedup, and a larger speedup is possible if you're okay with the output being numpy arrays instead of lists. As is:
```

```python
%timeit new_function2(close, volume, INTERVAL_LENGTH)
1 loops, best of 3: 1.15 s per loop
```
You can replace your inner loop with a call to np.cumsum()... See my "new_function" function below. This gives a considerable speedup...
```
```

```python
%timeit new_function(close, volume, INTERVAL_LENGTH)
1 loops, best of 3: 15.7 s per loop
```
vs
```
```

```python
%timeit old_function(close, volume, INTERVAL_LENGTH)
1 loops, best of 3: 53.1 s per loop
```
It should be possible to vectorize the entire thing and avoid for loops entirely, though... Give me an minute, and I'll see what I can do...
```
import numpy as np

ARRAY_LENGTH = 500000
INTERVAL_LENGTH = 15
close = np.arange(ARRAY_LENGTH, dtype=np.float)
volume = np.arange(ARRAY_LENGTH, dtype=np.float)

def old_function(close, volume, INTERVAL_LENGTH):
    results = []
    for i in xrange(len(close) - INTERVAL_LENGTH):
        for j in xrange(i+1, i+INTERVAL_LENGTH):
            ret = close[j] / close[i]
            vol = sum( volume[i+1:j+1] )
            if (ret > 1.0001) and (ret < 1.5) and (vol > 100):
                results.append( (i, j, ret, vol) )
    return results

def new_function(close, volume, INTERVAL_LENGTH):
    results = []
    for i in xrange(close.size - INTERVAL_LENGTH):
        vol = volume[i+1:i+INTERVAL_LENGTH].cumsum()
        ret = close[i+1:i+INTERVAL_LENGTH] / close[i]

        filter = (ret > 1.0001) & (ret < 1.5) & (vol > 100)
        j = np.arange(i+1, i+INTERVAL_LENGTH)[filter]

        tmp_results = zip(j.size * [i], j, ret[filter], vol[filter])
        results.extend(tmp_results)
    return results

def new_function2(close, volume, INTERVAL_LENGTH):
    vol, ret = [], []
    I, J = [], []
    for k in xrange(1, INTERVAL_LENGTH):
        start = k
        end = volume.size - INTERVAL_LENGTH + k
        vol.append(volume[start:end])
        ret.append(close[start:end])
        J.append(np.arange(start, end))
        I.append(np.arange(volume.size - INTERVAL_LENGTH))

    vol = np.vstack(vol)
    ret = np.vstack(ret)
    J = np.vstack(J)
    I = np.vstack(I)

    vol = vol.cumsum(axis=0)
    ret = ret / close[:-INTERVAL_LENGTH]

    filter = (ret > 1.0001) & (ret < 1.5) & (vol > 100)

    vol = vol[filter]
    ret = ret[filter]
    I = I[filter]
    J = J[filter]

    output = zip(I.flat,J.flat,ret.flat,vol.flat)
    return output

results = old_function(close, volume, INTERVAL_LENGTH)
results2 = new_function(close, volume, INTERVAL_LENGTH)
results3 = new_function(close, volume, INTERVAL_LENGTH)

# Using sets to compare, as the output
# is in a different order than the original function
print set(results) == set(results2)
print set(results) == set(results3)
```
```


Question:
Hilbert Transform in Python? I am trying to write the Hilbert transform from scratch but not use any built in libraries other than for fft and ifft. I am not a mathematician by trade but I found these two algorithms online for the Hilbert transform, one in C and one in MATLAB. I tried to do implement them both but neither of them are giving me the same result as SciPy's Hilbert. I for sure have made something wrong in my implementation. Any insight would be greatly appreciated. First implementation: (From MATLAB Website) Hilbert uses a four-step algorithm: Calculate the FFT of the input sequence, storing the result in a vector x. Create a vector h whose elements h(i) have the values: 1 for i = 1, (n/2)+1 2 for i = 2, 3, ... , (n/2) 0 for i = (n/2)+2, ... , n Calculate the element-wise product of x and h. Calculate the inverse FFT of the sequence obtained in step 3 and returns the first n elements of the result. My attempt:
```py
def generate_array(n):
    a = np.hstack((np.full(n//2+1, 2), np.zeros(n//2-1)))
    a[[0, n//2]] = 1
    return a

def hilbert_from_scratch_2(u):
    fft_result = fft(u) #scipy fft

    n = len(u)

    to_multiply = generate_array(n)

    result = np.multiply(n,to_multiply)

    return ifft(result) #scipy ifft
```
Second Implementation: (https://www.cfa.harvard.edu/~spaine/am/download/src/transform.c)
```c
void hilbert(double *z, unsigned long n)
{
    double x;
    unsigned long i, n2;

    n2 = n << 1;
    /*
     * Compute the (bit-reversed) Fourier transform of z.
     */
    fft_dif(z, n);
    /*
     * Form the transform of the analytic sequence by zeroing
     * the transform for negative time, except for the (N/2)th.
     * element.  Since z is now in bit-reversed order, this means
     * zeroing every other complex element.  The array indices of
     * the elements to be zeroed are 6,7,10,11...etc. (The real
     * and imaginary parts of the (N/2)th element are in z[2] and
     * z[3], respectively.)
     */
    for (i = 6; i < n2; i += 4) {
        z[i] = 0.;
        z[i+1] = 0.;
    }
    /*
     * The 0th and (N/2)th elements get multiplied by 0.5.  Test
     * for the trivial 1-point transform, just in case.
     */
    z[0] *= 0.5;
    z[1] *= 0.5;
    if (n > 1) {
        z[2] *= 0.5;
        z[3] *= 0.5;
    }
    /*
     * Compute the inverse transform.
     */
    ifft_dit(z, n);
    /*
     * Normalize the array.  The factor of 2 is left over from
     * forming the transform in the time domain.
     */
    x = 2. / (double)n;
    for (i = 0; i < n2; ++i)
        z[i] *= x;
    return;
}   /* hilbert() */
```
My attempt:
```py
def hilbert_from_scratch(signal):
    fast_ft = fft(signal) #scipy fft
    for i in range(6,len(signal),4):
        fast_ft[i] = 0
        fast_ft[i+1] = 0

    fast_ft[0] = fast_ft[0]*.5
    fast_ft[1] = fast_ft[1]*.5

    if(len(fast_ft) > 1):
        fast_ft[2] = fast_ft[2]*.5
        fast_ft[3] = fast_ft[3]*.5

    inverse_fft = ifft(fast_ft) #scipy ifft

    x = 2 / len(signal)

    for i in range(0,len(signal),1):
        inverse_fft[i] = inverse_fft[i]*x

    return inverse_fft
```
Any insight on why neither of them give the same result as SciPy's hilbert would be greatly appreciated.

Response:
I had a look at your code, made a couple edits, and compared it to the scipy and MATLAB Hilbert transforms. The function hilbert_from_scratch returns a complex sequence; the real components are the original signal and the complex components are the Hilbert transform. If you want just the Hilbert Transform, use np.imag on the returned array.
```py
import math
from scipy.fftpack import *

def hilbert_from_scratch(u):
    # N : fft length
    # M : number of elements to zero out
    # U : DFT of u
    # v : IDFT of H(U)

    N = len(u)
    # take forward Fourier transform
    U = fft(u)
    M = N - N//2 - 1
    # zero out negative frequency components
    U[N//2+1:] = [0] * M
    # double fft energy except @ DC0
    U[1:N//2] = 2 * U[1:N//2]
    # take inverse Fourier transform
    v = ifft(U)
    return v

if __name__ == '__main__':
    N = 32
    f = 1
    dt = 1.0 / N
    y = []
    for n in range(N):
        x = 2*math.pi*f*dt*n
        y.append(math.sin(x))
    z1 = hilbert_from_scratch(y)
    z2 = hilbert(y)
    print(" n     y fromscratch scipy")
    for n in range(N):
        print('{:2d} {:+5.2f} {:+10.2f} {:+5.2f}'.format(n, y[n], z1[n], z2[n]))
```
Outputs:
```
n     y fromscratch scipy
 0 +0.00 +0.00-1.00j +1.00
 1 +0.38 +0.38-0.92j +0.92
 2 +0.71 +0.71-0.71j +0.71
 3 +0.92 +0.92-0.38j +0.38
 4 +1.00 +1.00-0.00j +0.00
 5 +0.92 +0.92+0.38j -0.38
 6 +0.71 +0.71+0.71j -0.71
 7 +0.38 +0.38+0.92j -0.92
 8 +0.00 +0.00+1.00j -1.00
 9 -0.38 -0.38+0.92j -0.92
10 -0.71 -0.71+0.71j -0.71
11 -0.92 -0.92+0.38j -0.38
12 -1.00 -1.00+0.00j -0.00
13 -0.92 -0.92-0.38j +0.38
14 -0.71 -0.71-0.71j +0.71
15 -0.38 -0.38-0.92j +0.92
```
MATLAB:
```
>> y = sin(2*pi*linspace(0,1,17)); z = hilbert(y(1:end-1));
>> fprintf('%+2.2f %+2.2f\n',[real(z);imag(z)])
+0.00 -1.00
+0.38 -0.92
+0.71 -0.71
+0.92 -0.38
+1.00 -0.00
+0.92 +0.38
+0.71 +0.71
+0.38 +0.92
+0.00 +1.00
-0.38 +0.92
-0.71 +0.71
-0.92 +0.38
-1.00 +0.00
-0.92 -0.38
-0.71 -0.71
-0.38 -0.92
```


Question:
Removing diagonal elements from a sparse matrix in scipy I want to remove diagonal elements from a sparse matrix. Since the matrix is sparse, these elements shouldn't be stored once removed. Scipy provides a method to set diagonal elements values: setdiag If I try it using lil_matrix, it works:
```

```python
a = np.ones((2,2))
```

```python
c = lil_matrix(a)
```

```python
c.setdiag(0)
```

```python
c
#Output
#<2x2 sparse matrix of type '<type 'numpy.float64'>'
#    with 2 stored elements in LInked List format>
#```
#However with csr_matrix, it seems diagonal elements are not removed from storage:
#```
```

```python
b = csr_matrix(a)
```

```python
b
#Output
#<2x2 sparse matrix of type '<type 'numpy.float64'>'
#    with 4 stored elements in Compressed Sparse Row format>
```

```python
b.setdiag(0)
```

```python
b
#Output
#<2x2 sparse matrix of type '<type 'numpy.float64'>'
#    with 4 stored elements in Compressed Sparse Row format>
```

```python
b.toarray()
#Output
#array([[ 0.,  1.],
#       [ 1.,  0.]])
#```
#Through a dense array, we have of course:
#```
```

```python
csr_matrix(b.toarray())
#Output
#<2x2 sparse matrix of type '<type 'numpy.float64'>'
#    with 2 stored elements in Compressed Sparse Row format>
#```
#Is that intended? If so, is it due to the compressed format of csr matrices? Is there any workaround else than going from sparse to dense to sparse again?
```

Response:
Simply setting elements to 0 does not change the sparsity of a csr matrix. You have to apply eliminate_zeros.
```

```python
a=sparse.csr_matrix(np.ones((2,2)))
```

```python
a
#Output
#<2x2 sparse matrix of type '<class 'numpy.float64'>'
#    with 4 stored elements in Compressed Sparse Row format>
```

```python
a.setdiag(0)
```

```python
a
#Output
#<2x2 sparse matrix of type '<class 'numpy.float64'>'
#    with 4 stored elements in Compressed Sparse Row format>
```

```python
a.eliminate_zeros()
```

```python
a
#Output
#<2x2 sparse matrix of type '<class 'numpy.float64'>'
#    with 2 stored elements in Compressed Sparse Row format>
#```
#Since changing the sparsity of a csr matrix is relatively expensive, they let you change values to 0 without changing sparsity.
#```
```

```python
%%timeit a=sparse.csr_matrix(np.ones((1000,1000)))
     ...: a.setdiag(0)
100 loops, best of 3: 3.86 ms per loop
```

```python
%%timeit a=sparse.csr_matrix(np.ones((1000,1000)))
     ...: a.setdiag(0)
     ...: a.eliminate_zeros()
SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
10 loops, best of 3: 133 ms per loop
```

```python
%%timeit a=sparse.lil_matrix(np.ones((1000,1000)))
     ...: a.setdiag(0)
100 loops, best of 3: 14.1 ms per loop
```
```


Question:
Scipy: Linear programming with sparse matrices I want to solve a linear program in python. The number of variables (I will call it N from now on) is very large (~50000) and in order to formulate the problem in the way scipy.optimize.linprog requires it, I have to construct two N x N matrices (A and B below). The LP can be written as
```
minimize: c.x
subject to:
    A.x <= a
    B.x  = b
    x_i >= 0 for all i in {0, ..., n}
```
whereby . denotes the dot product and a, b, and c are vectors with length N. My experience is that constructing such large matrices (A and B have both approx. 50000x50000 = 25*10^8 entries) comes with some issues: If the hardware is not very strong, NumPy may refuse to construct such big matrices at all (see for example Very large matrices using Python and NumPy) and even if NumPy creates the matrix without problems, there is a huge performance issue. This is natural regarding the huge amount of data NumPy has to deal with. However, even though my linear program comes with N variables, the matrices I work with are very sparse. One of them has only entries in the very first row, the other one only in the first M rows, with M < N/2. Of course I would like to exploit this fact. As far as I have read (e.g. Trying to solve a Scipy optimization problem using sparse matrices and failing), scipy.optimize.linprog does not work with sparse matrices. Therefore, I have the following questions: Is it actually true that SciPy does not offer any possibility to solve a linear program with sparse matrices? (If not, how can I do it?) Do you know any alternative library that will solve the problem more effectively than SciPy with non-sparse matrices? (The library suggested in the thread above seems to be not flexible enough for my purposes - as far as I understand its documentation) Can it be expected that a new implementation of the simplex algorithm (using plain Python, no C) that exploits the sparsity of the matrices will be more efficient than SciPy with non-sparse matrices?

Response: I would say forming a dense matrix (or two) to solve a large sparse LP is probably not the right thing to do. When solving a large sparse LP it is important to use a solver that has facilities to handle such problems and also to generate the model in a way that does not create explicitly any of these zero elements. Writing a stable, fast, sparse Simplex LP solver in Python as a replacement for the SciPy dense solver is not a trivial exercise. Moreover a solver written in pure Python may not perform as well. For the size you indicate, although not very, very large (may be large medium sized model would be a good classification) you may want to consider a commercial solver like Cplex, Gurobi or Mosek. These solvers are very fast and very reliable (they solve basically any LP problem you throw at them). They all have Python APIs. The solvers are free or very cheap for academics. If you want to use an Open Source solver, you may want to look at the COIN CLP solver. It also has a Python interface. If your model is more complex then you also may want to consider to use a Python modeling tool such as Pulp or Pyomo (Gurobi also has good modeling support in Python).


Question:
Bandpass Butterworth filter frequencies in scipy I'm designing a bandpass filter in scipy following the cookbook. However, if I decrecrease the filtering frequencies too much I end up with garbage at high order filters. What am I doing wrong?
```
from scipy.signal import butter, lfilter

def butter_bandpass(lowcut, highcut, fs, order=5):
    nyq = 0.5 * fs
    low = lowcut / nyq
    high = highcut / nyq
    b, a = butter(order, [low, high], btype='band')
    return b, a

if __name__ == "__main__":
    import numpy as np
    import matplotlib.pyplot as plt
    from scipy.signal import freqz
    # Sample rate and desired cutoff frequencies (in Hz).
    fs = 25
    # Plot the frequency response for a few different orders.
    plt.figure(1)
    plt.clf()
    for order in [1, 3, 5, 6, 9]:
        b, a = butter_bandpass(0.5, 4, fs, order=order)
        w, h = freqz(b, a, worN=2000)#np.logspace(-4, 3, 2000))
        plt.semilogx((fs * 0.5 / np.pi) * w, abs(h), label="order = %d" % order)
    plt.xlabel('Frequency (Hz)')
    plt.ylabel('Gain')
    plt.grid(True)
    plt.legend(loc='best')

    plt.figure(2)
    plt.clf()
    for order in [1, 3, 5, 6, 9]:
        b, a = butter_bandpass(0.05, 0.4, fs, order=order)
        w, h = freqz(b, a, worN=2000)#np.logspace(-4, 3, 2000))
        plt.semilogx((fs * 0.5 / np.pi) * w, abs(h), label="order = %d" % order)
    plt.xlabel('Frequency (Hz)')
    plt.ylabel('Gain')
    plt.grid(True)
    plt.legend(loc='best')

    plt.show()
```
Update: the issue was discussed and apparently solved on Scipy 0.14. However, the plot still looks really bad after Scipy update. What's wrong?

Response: Don't use b, a = butter for high-order filters, whether in Matlab or SciPy or Octave. Transfer function format has numerical stability problems, because some of the coefficients are very large while others are very small. This is why we changed the filter design functions to use zpk format internally. To see the benefits of this, you need to use z, p, k = butter(output='zpk') and then work with poles and zeros instead of numerator and denominator. Don't do high-order digital filters in a single stage. This is a bad idea no matter what software or hardware you're implementing them on. Typically it's best to break them up into second-order sections. In Matlab, you can use zp2sos to generate these automatically. In SciPy, you can use sos = butter(output='sos') and then filter using sosfilt() or sosfiltfilt(). This is the recommended way to filter for most applications.


Question:
Minimize quadratic function subject to linear equality constraints with SciPy I have a reasonably simple constrained optimization problem but get different answers depending on how I do it. Let's get the import and a pretty print function out of the way first:
```
import numpy as np
from scipy.optimize import minimize, LinearConstraint, NonlinearConstraint, SR1

def print_res( res, label ):
    print("\n\n ***** ", label, " ***** \n")
    print(res.message)
    print("obj func value at solution", obj_func(res.x))
    print("starting values: ", x0)
    print("ending values:   ", res.x.astype(int) )
    print("% diff", (100.*(res.x-x0)/x0).astype(int) )
    print("target achieved?",target,res.x.sum())
```
The sample data is very simple:
```
n = 5
x0 = np.arange(1,6) * 10_000
target = x0.sum() + 5_000   # increase sum from 15,000 to 20,000
```
Here's the constrained optimization (including jacobians). In words, the objective function I want to minimize is just the sum of squared percentage changes from the initial values to final values. The linear equality constraint is simply requiring x.sum() to equal a constant.
```
def obj_func(x):
    return ( ( ( x - x0 ) / x0 ) ** 2 ).sum()

def obj_jac(x):
    return 2. * ( x - x0 ) / x0 ** 2

def constr_func(x):
    return x.sum() - target

def constr_jac(x):
    return np.ones(n)
```
And for comparison, I've re-factored as an unconstrained minimization by using the equality constraint to replace x[0] with a function of x[1:]. Note that the unconstrained function is passed x0[1:] whereas the constrained function is passed x0.
```
def unconstr_func(x):
    x_one       = target - x.sum()
    first_term  = ( ( x_one - x0[0] ) / x0[0] ) ** 2
    second_term = ( ( ( x - x0[1:] ) / x0[1:] ) ** 2 ).sum()
    return first_term + second_term
```
I then try to minimize in three ways: Unconstrained with 'Nelder-Mead' Constrained with 'trust-constr' (w/ & w/o jacobian) Constrained with 'SLSQP' (w/ & w/o jacobian) Code:
```
##### (1) unconstrained

res0 = minimize( unconstr_func, x0[1:], method='Nelder-Mead')   # OK, but weird note
res0.x = np.hstack( [target - res0.x.sum(), res0.x] )
print_res( res0, 'unconstrained' )

##### (2a) constrained -- trust-constr w/ jacobian

nonlin_con = NonlinearConstraint( constr_func, 0., 0., constr_jac )
resTCjac = minimize( obj_func, x0, method='trust-constr',
                     jac='2-point', hess=SR1(), constraints = nonlin_con )
print_res( resTCjac, 'trust-const w/ jacobian' )

##### (2b) constrained -- trust-constr w/o jacobian

nonlin_con = NonlinearConstraint( constr_func, 0., 0. )
resTC = minimize( obj_func, x0, method='trust-constr',
                  jac='2-point', hess=SR1(), constraints = nonlin_con )
print_res( resTC, 'trust-const w/o jacobian' )

##### (3a) constrained -- SLSQP w/ jacobian

eq_cons = { 'type': 'eq', 'fun' : constr_func, 'jac' : constr_jac }
resSQjac = minimize( obj_func, x0, method='SLSQP',
                     jac = obj_jac, constraints = eq_cons )
print_res( resSQjac, 'SLSQP w/ jacobian' )

##### (3b) constrained -- SLSQP w/o jacobian

eq_cons = { 'type': 'eq', 'fun' : constr_func }
resSQ = minimize( obj_func, x0, method='SLSQP',
                  jac = obj_jac, constraints = eq_cons )
print_res( resSQ, 'SLSQP w/o jacobian' )
```
Here is some simplified output (and of course you can run the code to get the full output):
```
starting values:  [10000 20000 30000 40000 50000]

***** (1) unconstrained  *****
Optimization terminated successfully.
obj func value at solution 0.0045454545454545305
ending values:    [10090 20363 30818 41454 52272]

***** (2a) trust-const w/ jacobian  *****
The maximum number of function evaluations is exceeded.
obj func value at solution 0.014635854609684874
ending values:    [10999 21000 31000 41000 51000]

***** (2b) trust-const w/o jacobian  *****
`gtol` termination condition is satisfied.
obj func value at solution 0.0045454545462939935
ending values:    [10090 20363 30818 41454 52272]

***** (3a) SLSQP w/ jacobian  *****
Optimization terminated successfully.
obj func value at solution 0.014636111111111114
ending values:    [11000 21000 31000 41000 51000]

***** (3b) SLSQP w/o jacobian  *****
Optimization terminated successfully.
obj func value at solution 0.014636111111111114
ending values:    [11000 21000 31000 41000 51000]
```
Notes: (1) & (2b) are plausible solutions in that they achieve significantly lower objective function values and intuitively we'd expect the variables with larger starting values to move more (both absolutely and in percentage terms) than the smaller ones. Adding the jacobian to 'trust-const' causes it to get the wrong answer (or at least a worse answer) and also to exceed max iterations. Maybe the jacobian is wrong, but the function is so simple that I'm pretty sure it's correct (?) 'SLSQP' doesn't seem to work w/ or w/o the jacobian supplied, but works very fast and claims to terminate successfully. This seems very worrisome in that getting the wrong answer and claiming to have terminated successfully is pretty much the worst possible outcome. Initially I used very small starting values and targets (just 1/1,000 of what I have above) and in that case all 5 approaches above work fine and give the same answers. My sample data is still extremely small, and it seems kinda bizarre for it to handle 1,2,..,5 but not 1000,2000,..5000. FWIW, note that the 3 incorrect results all hit the target by adding 1,000 to each initial value -- this satisfies the constraint but comes nowhere near minimizing the objective function (b/c variables with higher initial values should be increased more than lower ones to minimize the sum of squared percentage differences). So my question is really just what is happening here and why do only (1) and (2b) seem to work? More generally, I'd like to find a good python-based approach to this and similar optimization problems and will consider answers using other packages besides scipy although the best answer would ideally also address what is going on with scipy here (e.g. is this user error or a bug I should post to github?).

Response:
Here is how this problem could be solved using nlopt which is a library for nonlinear optimization which I've been pretty impressed with. First, the objective function and gradient are both defined using the same function:
```py
def obj_func(x, grad):
    if grad.size > 0:
        grad[:] = obj_jac(x)
    return ( ( ( x/x0 - 1 )) ** 2 ).sum()

def obj_jac(x):
    return 2. * ( x - x0 ) / x0 ** 2

def constr_func(x, grad):
    if grad.size > 0:
        grad[:] = constr_jac(x)
    return x.sum() - target

def constr_jac(x):
    return np.ones(n)
```
Then, to run the minimization using Nelder-Mead and SLSQP:
```py
opt = nlopt.opt(nlopt.LN_NELDERMEAD,len(x0)-1)
opt.set_min_objective(unconstr_func)
opt.set_ftol_abs(1e-15)
xopt = opt.optimize(x0[1:].copy())
xopt = np.hstack([target - xopt.sum(), xopt])
fval = opt.last_optimum_value()
print_res(xopt,fval,"Nelder-Mead");

opt = nlopt.opt(nlopt.LD_SLSQP,len(x0))
opt.set_min_objective(obj_func)
opt.add_equality_constraint(constr_func)
opt.set_ftol_abs(1e-15)
xopt = opt.optimize(x0.copy())
fval = opt.last_optimum_value()
print_res(xopt,fval,"SLSQP w/ jacobian");
```
And here are the results:
```
*****  Nelder-Mead  *****

obj func value at solution 0.00454545454546
result:  3
starting values:  [ 10000.  20000.  30000.  40000.  50000.]
ending values:    [10090 20363 30818 41454 52272]
% diff [0 1 2 3 4]
target achieved? 155000.0 155000.0

 *****  SLSQP w/ jacobian  *****

obj func value at solution 0.00454545454545
result:  3
starting values:  [ 10000.  20000.  30000.  40000.  50000.]
ending values:    [10090 20363 30818 41454 52272]
% diff [0 1 2 3 4]
target achieved? 155000.0 155000.0
```
When testing this out, I think I discovered what the issue with the original attempt was. If I set the absolute tolerance on the function to 1e-8 which is what the scipy functions default to I get:
```
*****  Nelder-Mead  *****

obj func value at solution 0.0045454580693
result:  3
starting values:  [ 10000.  20000.  30000.  40000.  50000.]
ending values:    [10090 20363 30816 41454 52274]
% diff [0 1 2 3 4]
target achieved? 155000.0 155000.0

 *****  SLSQP w/ jacobian  *****

obj func value at solution 0.0146361108503
result:  3
starting values:  [ 10000.  20000.  30000.  40000.  50000.]
ending values:    [10999 21000 31000 41000 51000]
% diff [9 5 3 2 2]
target achieved? 155000.0 155000.0
```
which is exactly what you were seeing. So my guess is that the minimizer ends up somewhere in the likelihood space during SLSQP where the next jump is less than 1e-8 from the last place.


Question:
finding nearest items across two lists/arrays in Python I have two numpy arrays x and y containing float values. For each value in x, I want to find the closest element in y, without reusing elements from y. The output should be a 1-1 mapping of indices of elements from x to indices of elements from y. Here's a bad way to do it that relies on sorting. It removes each element that was paired from the list. Without sorting this would be bad because the pairing would depend on the order of the original input arrays.
```
def min_i(values):
    min_index, min_value = min(enumerate(values),
                               key=operator.itemgetter(1))
    return min_index, min_value

# unsorted elements
unsorted_x = randn(10)*10
unsorted_y = randn(10)*10

# sort lists
x = sort(unsorted_x)
y = sort(unsorted_y)

pairs = []
indx_to_search = range(len(y))

for x_indx, x_item in enumerate(x):
    if len(indx_to_search) == 0:
        print "ran out of items to match..."
        break
    # until match is found look for closest item
    possible_values = y[indx_to_search]
    nearest_indx, nearest_item = min_i(possible_values)
    orig_indx = indx_to_search[nearest_indx]
    # remove it
    indx_to_search.remove(orig_indx)
    pairs.append((x_indx, orig_indx))
print "paired items: "
for k,v in pairs:
    print x[k], " paired with ", y[v]
```
I prefer to do it without sorting the elements first, but if they are sorted then I want to get the indices in the original, unsorted lists unsorted_x, unsorted_y. what's the best way to do this in numpy/scipy/Python or using pandas? thanks. edit: to clarify I'm not trying to find the best fit across all elemets (not minimizing sum of distances for example) but rather the best fit for each element, and it's okay if it's sometimes at the expense of other elements. I assume that y is generally much larger than x contrary to above example and so there's usually many very good fits for each value of x in y, and I just want to find that one efficiently. can someone show an example of scipy kdtrees for this? The docs are quite sparse
```
kdtree = scipy.spatial.cKDTree([x,y])
kdtree.query([-3]*10) # ?? unsure about what query takes as arg
```

Response:
EDIT 2 A solution using KDTree can perform very well if you can choose a number of neighbors that guarantees that you will have a unique neighbor for every item in your array. With the following code:
```
def nearest_neighbors_kd_tree(x, y, k) :
    x, y = map(np.asarray, (x, y))
    tree =scipy.spatial.cKDTree(y[:, None])
    ordered_neighbors = tree.query(x[:, None], k)[1]
    nearest_neighbor = np.empty((len(x),), dtype=np.intp)
    nearest_neighbor.fill(-1)
    used_y = set()
    for j, neigh_j in enumerate(ordered_neighbors) :
        for k in neigh_j :
            if k not in used_y :
                nearest_neighbor[j] = k
                used_y.add(k)
                break
    return nearest_neighbor
```
and a sample of n=1000 points, I get:
```

```python
np.any(nearest_neighbors_kd_tree(x, y, 12) == -1)
#Output
#True
```

```python
np.any(nearest_neighbors_kd_tree(x, y, 13) == -1)
#Output
#False
#```
#So the optimum is k=13, and then the timing is:
#```
```

```python
%timeit nearest_neighbors_kd_tree(x, y, 13)
100 loops, best of 3: 9.26 ms per loop
```
But in the worst case, you could need k=1000, and then:
```
```

```python
%timeit nearest_neighbors_kd_tree(x, y, 1000)
1 loops, best of 3: 424 ms per loop
```
Which is slower than the other options:
```
```

```python
%timeit nearest_neighbors(x, y)
10 loops, best of 3: 60 ms per loop
```

```python
%timeit nearest_neighbors_sorted(x, y)
10 loops, best of 3: 47.4 ms per loop
```
EDIT Sorting the array before searching pays off for arrays of more than 1000 items:
```
def nearest_neighbors_sorted(x, y) :
    x, y = map(np.asarray, (x, y))
    y_idx = np.argsort(y)
    y = y[y_idx]
    nearest_neighbor = np.empty((len(x),), dtype=np.intp)
    for j, xj in enumerate(x) :
        idx = np.searchsorted(y, xj)
        if idx == len(y) or idx != 0 and y[idx] - xj > xj - y[idx-1] :
            idx -= 1
        nearest_neighbor[j] = y_idx[idx]
        y = np.delete(y, idx)
        y_idx = np.delete(y_idx, idx)
    return nearest_neighbor
```
With a 10000 element long array:
```
```

```python
%timeit nearest_neighbors_sorted(x, y)
1 loops, best of 3: 557 ms per loop
```

```python
%timeit nearest_neighbors(x, y)
1 loops, best of 3: 1.53 s per loop
```
For smaller arrays it performs slightly worse. You are going to have to loop over all your items to implement your greedy nearest neighbor algorithm, if only to discard duplicates. With that in mind, this is the fastest I have been able to come up with:
```
def nearest_neighbors(x, y) :
    x, y = map(np.asarray, (x, y))
    y = y.copy()
    y_idx = np.arange(len(y))
    nearest_neighbor = np.empty((len(x),), dtype=np.intp)
    for j, xj in enumerate(x) :
        idx = np.argmin(np.abs(y - xj))
        nearest_neighbor[j] = y_idx[idx]
        y = np.delete(y, idx)
        y_idx = np.delete(y_idx, idx)

    return nearest_neighbor
```
And now with:
```
n = 1000
x = np.random.rand(n)
y = np.random.rand(2*n)
```
I get:
```
```

```python
%timeit nearest_neighbors(x, y)
10 loops, best of 3: 52.4 ms per loop
```
```


Question:
Scipy: Minimize violates given bounds I am trying to use scipy.optimize.minimize with simple a <= x <= b bounds. However, it often happens that my target function is evaluated just outside the bounds. To my understanding, this happens when minimize tries to determine the gradient of the target function at the boundary. Minimal example:
```
import math
import numpy as np
from scipy.optimize import Bounds, minimize

constraint = Bounds([-1, -1], [1, 1], True)
def fun(x):
    print(x)
    return -math.exp(-np.dot(x,x))
result = minimize(fun, [-1, -1], bounds=constraint)
```
The output shows that the minimizer jumps to the point [1,1] and then tries to evaluate at [1.00000001, 1]:
```
[-1. -1.]
[-0.99999999 -1.        ]
[-1.         -0.99999999]
[-0.72932943 -0.72932943]
[-0.72932942 -0.72932943]
[-0.72932943 -0.72932942]
[-0.22590689 -0.22590689]
[-0.22590688 -0.22590689]
[-0.22590689 -0.22590688]
[1. 1.]
[1.00000001 1.        ]
[1.         1.00000001]
[-0.03437328 -0.03437328]
...
```
Of course, there is no problem in this example, as fun can be evaluated also there. But that might not always be the case... In my actual problem, the minimum can not be on the boundary and I have the easy workaround of adding an epsilon to the bounds. But one would expect that there should be an easy solution to this issue which also works if the minimum can be at a boundary? PS: It would be strange if I were the first to have this problem -- sorry if this question has been asked before somewhere, but I didn't find it anywhere.

Response:
As discussed here (thanks @"Welcome to Stack Overflow" for the comment directing me there), the problem is indeed that the gradient routine doesn't respect the bounds. I wrote a new one that does the job:
```
import math
import numpy as np
from scipy.optimize import minimize

def gradient_respecting_bounds(bounds, fun, eps=1e-8):
    """bounds: list of tuples (lower, upper)"""
    def gradient(x):
        fx = fun(x)
        grad = np.zeros(len(x))
        for k in range(len(x)):
            d = np.zeros(len(x))
            d[k] = eps if x[k] + eps <= bounds[k][1] else -eps
            grad[k] = (fun(x + d) - fx) / d[k]
        return grad
    return gradient

bounds = ((-1, 1), (-1, 1))
def fun(x):
    print(x)
    return -math.exp(-np.dot(x,x))
result = minimize(fun, [-1, -1], bounds=bounds,
                  jac=gradient_respecting_bounds(bounds, fun))
```
Note that this can be a bit less efficient, because fun(x) now gets evaluated twice at each point. This seems to be unavoidable, relevant snippet from _minimize_lbfgsb in lbfgsb.py:
```
if jac is None:
    def func_and_grad(x):
        f = fun(x, *args)
        g = _approx_fprime_helper(x, fun, epsilon, args=args, f0=f)
        return f, g
else:
    def func_and_grad(x):
        f = fun(x, *args)
        g = jac(x, *args)
        return f, g
```
As you can see, the value of f can only be reused by the internal _approx_fprime_helper function.


Question:
Voronoi Diagram Edges: How to get edges in the form (point1, point2) from a scipy.spatial.Voronoi object? I have spent a great deal of time now trying to obtain edges from the scipy.spatial.Voronoi diagram to no avail. Here is the main documentation: http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.spatial.Voronoi.html If you create a Voronoi Diagram like so:
```
points = np.array([[0, 0], [0, 1], [0, 2], [1, 0], [1, 1], [1, 2],
[2, 0], [2, 1], [2, 2]]) //Or feel free to use any set of points
```
then you have access to the following object properties:
```
vor.regions
vor.max_bound
vor.ndim
vor.ridge_dict
vor.ridge_points
vor.ridge_vertices
vor.npoints
vor.point_region
vor.points
```
But is unclear how to combine these to get edges in the form (point1, point2) for 2d voronoi diagrams? I know edges exist because you can plot the voronoi diagram and its edgres and vertices because you can do the following:
```
voronoi_plot_2d(vor)
plt.show()
```
which clearly depicts voronoi edgres - how to get a list of them and their starting and endpoints? Its okay if I only get the solid edges (not the dotted ones which go unbounded off the plot)

Response:
Take a look at the ridge_vertices attribute:
```
ridge_vertices  (list of list of ints, shape (nridges, *))
        Indices of the Voronoi vertices forming each Voronoi ridge.
```
Each element in that list is a pair of integers. Each integer is an index into the vertices list. So each element defines a line to be draw in the Voronoi diagram. An index of -1 means a point that is "at infinity". Here's script that draws the lines of the Voronoi diagram:
```
import numpy as np
import matplotlib.pyplot as plt
from scipy.spatial import Voronoi

points = np.array([[0, 0], [0, 1], [0, 2],
                   [1, 0], [1, 1], [1, 2],
                   [2, 0], [2, 1], [2, 2]])

vor = Voronoi(points)

fig = plt.figure()

# Mark the Voronoi vertices.
plt.plot(vor.vertices[:,0], vor.vertices[:, 1], 'ko', ms=8)

for vpair in vor.ridge_vertices:
    if vpair[0] >= 0 and vpair[1] >= 0:
        v0 = vor.vertices[vpair[0]]
        v1 = vor.vertices[vpair[1]]
        # Draw a line from v0 to v1.
        plt.plot([v0[0], v1[0]], [v0[1], v1[1]], 'k', linewidth=2)

plt.show()
```
It creates:


Question:
Readability of Scientific Python Code (Line Continuations, Variable Names, Imports) Do Python's stylistic best practices apply to scientific coding? I am finding it difficult to keep scientific Python code readable. For example, it is suggested to use meaningful names for variables and to keep the namespace ordered by avoiding import *. Thus, e.g. :
```
import numpy as np
    normbar = np.random.normal(mean, std, np.shape(foo))
```
But these suggestions can lead to some difficult-to-read code, especially given the 79-character line width. For example, I just wrote the following operation:
```
net["weights"][ix1][ix2] += lrate * (CD / nCases - opts["weightcost_pretrain"].dot(net["weights"][ix1][ix2]))
```
I can span the expression across lines:
```
net["weights"][ix1][ix2] += lrate * (CD / nCases -
     opts["weightcost_pretrain"].dot(net["weights"][ix1][ix2]))
```
but this does not seem much better, and I am not sure how deep to indent the second line. These kinds of line continuations become even trickier when one is double-indented into a nested loop, and there are only 50 characters available on a line. Should I accept that scientific Python looks clunky, or are there ways to avoid lines like the example above? Some potential approaches are: using shorter variable names using shorter dictionary key names importing numpy functions directly and assigning them short names defining helper functions for combinations of arithmetic operations breaking operations into smaller pieces, and placing one on each line I would appreciate any wisdom on which of these to pursue and which to avoid, as well as suggestions for other remedies.

Response:
defining helper functions for combinations of arithmetic operations breaking operations into smaller pieces, and placing one on each line These are both good ideas—in keeping with the intent behind PEP 8, and with Pythonic style in general. In fact, whenever someone suggests modifying PEP 8 to give more information about long lines, half the responses are usually "If you're going over the line limit, you're probably doing too much in one expression". And, more generally, factoring out code and giving sensible names to sensible operations are always a good idea. Of course without knowing exactly what all these things represent, I can only guess at how to split them up, but I think something like this would be pretty readable and meaningful:
```
cost = opts["weightcost_pretrain"].dot(net["weights"][ix1][ix2])
weight = lrate * (CD / nCases - cost)
net["weights"][ix1][ix2] += weight
```


Question: Share SciPy Sparse Array Between Process Objects I've recently been learning Python multiprocessing, and have run into a roadblock. I have a lerge sparse SciPy array (CSC-format), that I need to share in read only format between 5 worker-processes. I've read this and this (numpy-shared), but this seems to be only for dense-types. How would I share a scipy.sparse.csc_matrix() without copying (or with minimal copying) between 5 multiprocessing Process objects? Even the numpy-shared method seems to require copying the entire array, and even then, I can't just convert a scipy.sparse into a mp.Array(). Could anyone help point me in the right direction? Thanks!

Response:
I cannot help you with the multiprocessing part of your question, but a CSC sparse matrix is little more than three numpy arrays. You can instantiate another sparse matrix, b, sharing the same memory objects as a sparse matrix, a, by doing:
```
import scipy.sparse as sps

b = sps.csc_matrix((a.data, a.indices, a.indptr), shape=a.shape, copy=False)
```
a.data, a.indices and a.indptr are the three numpy arrays you want to share between your processes, if you can do that, then instantiating a sparse matrix in each process will be an inexpensive operation.


Question:
Integrating a function with singularities using scipy's quad routine I am using the quad function from scipy.integrate v0.19.1 to integrate functions with a square-root like singularity at each end of the integration interval such as for example
```

```python
quad(lambda x: 1/sqrt(1-x**2), -1, 1)
```
(I use the sqrt function from numpy v1.12.0) which immediately yields the correct result pi:
```
#Output
#(3.141592653589591, 6.200897573194197e-10)
#```
#According to the documentation of the quad function the keyword points should be used to indicate the locations of singularities or discontinuities of the integrand, but if I indicate the points [1, -1] where the above integrand is singluar I get a Warning and nan as result:
#```
```

```python
quad(lambda x: 1/sqrt(1-x**2), -1, 1, points=[-1, 1])

RuntimeWarning: divide by zero encountered in double_scalars
IntegrationWarning: Extremely bad integrand behavior occurs at some
points of the integration interval.
#Output
#(nan, nan)
#```
#Can someone clarify, why quad produces these problems if the singularities of the integrand are specified and just runs fine if these points are not indicated? EDIT: I think I figured out the correct solution for this problem. For the case someone else encounters similar problems I quickly want to share my findings: I you want to integrate a function of the form f(x)*g(x) with a smooth function f(x) and g(x) = (x-a)**alpha * (b-x)**beta, where a and b are the the integration limits and g(x) has singularities at these limits if alpha, beta < 0, then you are supposed to just integrate f(x) using g(x) as weighting function. For the quad routine, this is possible using the weight and wvar arguments. With these arguments you are also able to handle singularities of different kinds and problematic oscillatory behavior. The weighting function g(x) defined above can be used by setting weight='alg' and using wvar=(alpha, beta) to specify the exponents in g(x). Since 1/sqrt(1-x**2) = (x+1)**(-1/2) * (1-x)**(-1/2) I can now handle the integral as follows:
#```
```

```python
quad(lambda x: 1, -1, 1, weight='alg', wvar=(-1/2, -1/2))
#Output
#(3.1415926535897927, 9.860180640534107e-14)
#```
#which yields the correct answer pi with a very high accuracy, no matter if I use the argument points=(-1, 1) or not (which, as far as I understand now, should only be used, if the singularities/discontinuities can not be handled by choosing an appropriate weighting function).
```

Response:
The parameter points is meant for singularities/discontinuities that occur within the interval of integration. It is not intended for endpoints of the interval. So, in your example the version without points is the correct approach. (I can't pinpoint what goes wrong when endpoints are included in points without diving into FORTRAN code that SciPy wraps.) Compare with the following example, where singularities occur within the interval of integration:
```

```python
quad(lambda x: 1/sqrt(abs(1-x**2)), -2, 2)
#Output
#(inf, inf)
```

```python
quad(lambda x: 1/sqrt(abs(1-x**2)), -2, 2, points = [-1, 1])
#Output
#(5.775508447436837, 7.264979728915932e-10)
#```
#Here the inclusion of points is appropriate, and yields the correct result, while without points the output is worthless.
```


Question: Calculate the Cumulative Distribution Function (CDF) in Python How can I calculate in python the Cumulative Distribution Function (CDF)? I want to calculate it from an array of points I have (discrete distribution), not with the continuous distributions that, for example, scipy has.

Response:
(It is possible that my interpretation of the question is wrong. If the question is how to get from a discrete PDF into a discrete CDF, then np.cumsum divided by a suitable constant will do if the samples are equispaced. If the array is not equispaced, then np.cumsum of the array multiplied by the distances between the points will do.) If you have a discrete array of samples, and you would like to know the CDF of the sample, then you can just sort the array. If you look at the sorted result, you'll realize that the smallest value represents 0% , and largest value represents 100 %. If you want to know the value at 50 % of the distribution, just look at the array element which is in the middle of the sorted array. Let us have a closer look at this with a simple example:
```
import matplotlib.pyplot as plt
import numpy as np

# create some randomly ddistributed data:
data = np.random.randn(10000)

# sort the data:
data_sorted = np.sort(data)

# calculate the proportional values of samples
p = 1. * np.arange(len(data)) / (len(data) - 1)

# plot the sorted data:
fig = plt.figure()
ax1 = fig.add_subplot(121)
ax1.plot(p, data_sorted)
ax1.set_xlabel('$p$')
ax1.set_ylabel('$x$')

ax2 = fig.add_subplot(122)
ax2.plot(data_sorted, p)
ax2.set_xlabel('$x$')
ax2.set_ylabel('$p$')
```
This gives the following plot where the right-hand-side plot is the traditional cumulative distribution function. It should reflect the CDF of the process behind the points, but naturally, it is not as long as the number of points is finite. This function is easy to invert, and it depends on your application which form you need.


Question: Filling gaps in a numpy array I just want to interpolate, in the simplest possible terms, a 3D dataset. Linear interpolation, nearest neighbour, all that would suffice (this is to start off some algorithm, so no accurate estimate is required). In new scipy versions, things like griddata would be useful, but currently I only have scipy 0.8. So I have a "cube" (data[:,:,:], (NixNjxNk)) array, and an array of flags (flags[:,:,:,], True or False) of the same size. I want to interpolate my data for the elements of data where the corresponding element of flag is False, using eg the nearest valid datapoint in data, or some linear combination of "close by" points. There can be large gaps in the dataset in at least two dimensions. Other than coding a full-blown nearest neighbour algorithm using kdtrees or similar, I can't really find a generic, N-dimensional nearest-neighbour interpolator.

Response:
Using scipy.ndimage, your problem can be solved with nearest neighbor interpolation in 2 lines :
```
from scipy import ndimage as nd

indices = nd.distance_transform_edt(invalid_cell_mask, return_distances=False, return_indices=True)
data = data[tuple(ind)]
```
Now, in the form of a function:
```
import numpy as np
from scipy import ndimage as nd

def fill(data, invalid=None):
    """
    Replace the value of invalid 'data' cells (indicated by 'invalid')
    by the value of the nearest valid data cell

    Input:
        data:    numpy array of any dimension
        invalid: a binary array of same shape as 'data'.
                 data value are replaced where invalid is True
                 If None (default), use: invalid  = np.isnan(data)

    Output:
        Return a filled array.
    """
    if invalid is None: invalid = np.isnan(data)

    ind = nd.distance_transform_edt(invalid,
                                    return_distances=False,
                                    return_indices=True)
    return data[tuple(ind)]
```
Exemple of use:
```
def test_fill(s,d):
     # s is size of one dimension, d is the number of dimension
    data = np.arange(s**d).reshape((s,)*d)
    seed = np.zeros(data.shape,dtype=bool)
    seed.flat[np.random.randint(0,seed.size,int(data.size/20**d))] = True

    return fill(data,-seed), seed

import matplotlib.pyplot as plt
data,seed  = test_fill(500,2)
data[nd.binary_dilation(seed,iterations=2)] = 0   # draw (dilated) seeds in black
plt.imshow(np.mod(data,42))                       # show cluster
```
result:


Question:
Python interpolate point value on 2D grid I have a regular 2D X, Y and Z array and I have a point X0 and Y0 and I want to know the Z0 value in point (X0, Y0) on my grid. I found that scipy have interpolate module but as I understand it interpolates 1D/2D arrays and returns 1D/2D array, but there is no method that returns only one value at one point. For example:
```
#My grid data
X = [ [X11, X12, X13, ..., X1N],
      [X21, X22, X23, ..., X2N],
          ....
      [XN1, XN2, XN3, ..., XNN]

Y = [ [Y11, Y12, Y13, ..., Y1N],
      [Y21, Y22, Y23, ..., Y2N],
          ....
      [YN1, YN2, YN3, ..., YNN] ]

Z = [ [Z11, Z12, Z13, ..., Z1N],
      [Z21, Z22, Z23, ..., Z2N],
          ....
      [ZN1, ZN2, ZN3, ..., ZNN] ]

#Point at which I want to know the value of the Z
X0, Y0 = ..., ...

#Now I want to call any function that'll return the value at point (X0, Y0), Z0 is float value, not array
Z0 = interpolation(X, Y, Z, X0, Y0)
```
As I understand the similar function is scipy.interpolate.interpn but it works only with 1D arrays and give out an error when I want to work with 2D data

Response:
you can also use griddata :
```
points = np.array( (X.flatten(), Y.flatten()) ).T
values = Z.flatten()

from scipy.interpolate import griddata
Z0 = griddata( points, values, (X0,Y0) )
```
X0 and Y0 can be arrays or even a grid. you can also choose the interpolation with method= perhaps you can find a way to get ride of the flatten(), but it should work. (https://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html)


Question:
Correlation coefficients and p values for all pairs of rows of a matrix I have a matrix data with m rows and n columns. I used to compute the correlation coefficients between all pairs of rows using np.corrcoef:
```
import numpy as np
data = np.array([[0, 1, -1], [0, -1, 1]])
np.corrcoef(data)
```
Now I would also like to have a look at the p-values of these coefficients. np.corrcoef doesn't provide these; scipy.stats.pearsonr does. However, scipy.stats.pearsonr does not accept a matrix on input. Is there a quick way how to compute both the coefficient and the p-value for all pairs of rows (arriving e.g. at two m by m matrices, one with correlation coefficients, the other with corresponding p-values) without having to manually go through all pairs?

Response:
I have encountered the same problem today. After half an hour of googling, I can't find any code in numpy/scipy library can help me do this. So I wrote my own version of corrcoef
```
import numpy as np
from scipy.stats import pearsonr, betai

def corrcoef(matrix):
    r = np.corrcoef(matrix)
    rf = r[np.triu_indices(r.shape[0], 1)]
    df = matrix.shape[1] - 2
    ts = rf * rf * (df / (1 - rf * rf))
    pf = betai(0.5 * df, 0.5, df / (df + ts))
    p = np.zeros(shape=r.shape)
    p[np.triu_indices(p.shape[0], 1)] = pf
    p[np.tril_indices(p.shape[0], -1)] = p.T[np.tril_indices(p.shape[0], -1)]
    p[np.diag_indices(p.shape[0])] = np.ones(p.shape[0])
    return r, p

def corrcoef_loop(matrix):
    rows, cols = matrix.shape[0], matrix.shape[1]
    r = np.ones(shape=(rows, rows))
    p = np.ones(shape=(rows, rows))
    for i in range(rows):
        for j in range(i+1, rows):
            r_, p_ = pearsonr(matrix[i], matrix[j])
            r[i, j] = r[j, i] = r_
            p[i, j] = p[j, i] = p_
    return r, p
```
The first version use the result of np.corrcoef, and then calculate p-value based on triangle-upper values of corrcoef matrix. The second loop version just iterating over rows, do pearsonr manually.
```
def test_corrcoef():
    a = np.array([
        [1, 2, 3, 4],
        [1, 3, 1, 4],
        [8, 3, 8, 5],
        [2, 3, 2, 1]])

    r1, p1 = corrcoef(a)
    r2, p2 = corrcoef_loop(a)

    assert np.allclose(r1, r2)
    assert np.allclose(p1, p2)
```
The test passed, they are the same.
```
def test_timing():
    import time
    a = np.random.randn(100, 2500)

    def timing(func, *args, **kwargs):
        t0 = time.time()
        loops = 10
        for _ in range(loops):
            func(*args, **kwargs)
        print('{} takes {} seconds loops={}'.format(
            func.__name__, time.time() - t0, loops))

    timing(corrcoef, a)
    timing(corrcoef_loop, a)

if __name__ == '__main__':
    test_corrcoef()
    test_timing()
```
The performance on my Macbook against 100x2500 matrix corrcoef takes 0.06608104705810547 seconds loops=10 corrcoef_loop takes 7.585600137710571 seconds loops=10


Question: Is there a difference between scipy.pi, numpy.pi, or math.pi? In a project using SciPy and NumPy, when should one use scipy.pi vs numpy.pi vs just math.pi? Is there a difference between these values?

Response:
```

```python
import math
```

```python
import numpy as np
```

```python
import scipy
```

```python
math.pi == np.pi == scipy.pi
#Output
#True
#```
#So it doesn't matter, they are all the same value. The only reason all three modules provide a pi value is so if you are using just one of the three modules, you can conveniently have access to pi without having to import another module. They're not providing different values for pi.
```


Question:
Converting a scipy coo_matrix to pytorch sparse tensor I have a coo_matrix:
```
from scipy.sparse import coo_matrix
coo = coo_matrix((3, 4), dtype = "int8")
```
That I want converted to a pytorch sparse tensor. According to the documentation https://pytorch.org/docs/master/sparse.html it should follow the coo format, but I cannot find a simple way to do the conversion. Any help would be greatly appreciated!

Response:
Using the data as in the Pytorch docs, it can be done simply using the attributes of the Numpy coo_matrix:
```
import torch
import numpy as np
from scipy.sparse import coo_matrix

coo = coo_matrix(([3,4,5], ([0,1,1], [2,0,2])), shape=(2,3))

values = coo.data
indices = np.vstack((coo.row, coo.col))

i = torch.LongTensor(indices)
v = torch.FloatTensor(values)
shape = coo.shape

torch.sparse.FloatTensor(i, v, torch.Size(shape)).to_dense()
```
Output
```
0 0 3
4 0 5
[torch.FloatTensor of size 2x3]
```


Question: Random Number from Histogram Suppose I create a histogram using scipy/numpy, so I have two arrays: one for the bin counts, and one for the bin edges. If I use the histogram to represent a probability distribution function, how can I efficiently generate random numbers from that distribution?

Response:
It's probably what np.random.choice does in @Ophion's answer, but you can construct a normalized cumulative density function, then choose based on a uniform random number:
```
from __future__ import division
import numpy as np
import matplotlib.pyplot as plt

data = np.random.normal(size=1000)
hist, bins = np.histogram(data, bins=50)

bin_midpoints = bins[:-1] + np.diff(bins)/2
cdf = np.cumsum(hist)
cdf = cdf / cdf[-1]
values = np.random.rand(10000)
value_bins = np.searchsorted(cdf, values)
random_from_cdf = bin_midpoints[value_bins]

plt.subplot(121)
plt.hist(data, 50)
plt.subplot(122)
plt.hist(random_from_cdf, 50)
plt.show()
```
A 2D case can be done as follows:
```
data = np.column_stack((np.random.normal(scale=10, size=1000),
                        np.random.normal(scale=20, size=1000)))
x, y = data.T
hist, x_bins, y_bins = np.histogram2d(x, y, bins=(50, 50))
x_bin_midpoints = x_bins[:-1] + np.diff(x_bins)/2
y_bin_midpoints = y_bins[:-1] + np.diff(y_bins)/2
cdf = np.cumsum(hist.ravel())
cdf = cdf / cdf[-1]

values = np.random.rand(10000)
value_bins = np.searchsorted(cdf, values)
x_idx, y_idx = np.unravel_index(value_bins,
                                (len(x_bin_midpoints),
                                 len(y_bin_midpoints)))
random_from_cdf = np.column_stack((x_bin_midpoints[x_idx],
                                   y_bin_midpoints[y_idx]))
new_x, new_y = random_from_cdf.T

plt.subplot(121, aspect='equal')
plt.hist2d(x, y, bins=(50, 50))
plt.subplot(122, aspect='equal')
plt.hist2d(new_x, new_y, bins=(50, 50))
plt.show()
```


Question:
Double the length of a python numpy array with interpolated new values I have an array of 5 numbers:
```
A = [10, 20, 40, 80, 110]
```
I need to create a new array with a 10nth length numbers. The extra numbers could be the average number between the two # of A. for example: EDIT B = [10 , 15 , 20 ,30, 40, 60, 80, 95, 110 ] Is it possible using a scipy or numpy function ?

Response:
Use numpy.interp:
```
import numpy as np
Y = [10, 20, 40, 80, 110]
N = len(Y)
X = np.arange(0, 2*N, 2)
X_new = np.arange(2*N-1)       # Where you want to interpolate
Y_new = np.interp(X_new, X, Y)
print(Y_new)
```
yields
```
[  10.   15.   20.   30.   40.   60.   80.   95.  110.]
```


Question:
How to get the indices list of all NaN value in numpy array? Say now I have a numpy array which is defined as,
```
[[1,2,3,4],
[2,3,NaN,5],
[NaN,5,2,3]]
```
Now I want to have a list that contains all the indices of the missing values, which is [(1,2),(2,0)] at this case. Is there any way I can do that?

Response:
np.isnan combined with np.argwhere
```
x = np.array([[1,2,3,4],
              [2,3,np.nan,5],
              [np.nan,5,2,3]])
np.argwhere(np.isnan(x))
```
output:
```
array([[1, 2],
       [2, 0]])
```


Question:
Most efficient way to calculate radial profile I need to optimize this part of an image processing application. It is basically the sum of the pixels binned by their distance from the central spot.
```
def radial_profile(data, center):
    y,x = np.indices((data.shape)) # first determine radii of all pixels
    r = np.sqrt((x-center[0])**2+(y-center[1])**2)
    ind = np.argsort(r.flat) # get sorted indices
    sr = r.flat[ind] # sorted radii
    sim = data.flat[ind] # image values sorted by radii
    ri = sr.astype(np.int32) # integer part of radii (bin size = 1)
    # determining distance between changes
    deltar = ri[1:] - ri[:-1] # assume all radii represented
    rind = np.where(deltar)[0] # location of changed radius
    nr = rind[1:] - rind[:-1] # number in radius bin
    csim = np.cumsum(sim, dtype=np.float64) # cumulative sum to figure out sums for each radii bin
    tbin = csim[rind[1:]] - csim[rind[:-1]] # sum for image values in radius bins
    radialprofile = tbin/nr # the answer
    return radialprofile

img = plt.imread('crop.tif', 0)
# center, radi = find_centroid(img)
center, radi = (509, 546), 55
rad = radial_profile(img, center)

plt.plot(rad[radi:])
plt.show()
```
Input image: The radial profile: By extracting the peaks of the resulting plot, I can accurately find the radii of the outer rings, which is the end goal here. Edit: For further reference I'll post my final solution of this. Using cython I got about a 15-20x speed up compared to the accepted answer.
```
import numpy as np
cimport numpy as np
cimport cython
from cython.parallel import prange
from libc.math cimport sqrt, ceil

DTYPE_IMG = np.uint8
ctypedef np.uint8_t DTYPE_IMG_t
DTYPE = np.int
ctypedef np.int_t DTYPE_t

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.nonecheck(False)
cdef void cython_radial_profile(DTYPE_IMG_t [:, :] img_view, DTYPE_t [:] r_profile_view, int xs, int ys, int x0, int y0) nogil:

    cdef int x, y, r, tmp

    for x in prange(xs):
        for y in range(ys):
            r =<int>(sqrt((x - x0)**2 + (y - y0)**2))
            tmp = img_view[x, y]
            r_profile_view[r] +=  tmp

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.nonecheck(False)
def radial_profile(np.ndarray img, int centerX, int centerY):

    cdef int xs, ys, r_max
    xs, ys = img.shape[0], img.shape[1]

    cdef int topLeft, topRight, botLeft, botRight

    topLeft = <int> ceil(sqrt(centerX**2 + centerY**2))
    topRight = <int> ceil(sqrt((xs - centerX)**2 + (centerY)**2))
    botLeft = <int> ceil(sqrt(centerX**2 + (ys-centerY)**2))
    botRight = <int> ceil(sqrt((xs-centerX)**2 + (ys-centerY)**2))

    r_max = max(topLeft, topRight, botLeft, botRight)

    cdef np.ndarray[DTYPE_t, ndim=1] r_profile = np.zeros([r_max], dtype=DTYPE)
    cdef DTYPE_t [:] r_profile_view = r_profile
    cdef DTYPE_IMG_t [:, :] img_view = img

    with nogil:
        cython_radial_profile(img_view, r_profile_view, xs, ys, centerX, centerY)
    return r_profile
```

Response:
It looks like you could use numpy.bincount here:
```
import numpy as np

def radial_profile(data, center):
    y, x = np.indices((data.shape))
    r = np.sqrt((x - center[0])**2 + (y - center[1])**2)
    r = r.astype(np.int)

    tbin = np.bincount(r.ravel(), data.ravel())
    nr = np.bincount(r.ravel())
    radialprofile = tbin / nr
    return radialprofile
```


Question: Convert an image RGB->Lab with python What is the preferred way of doing the conversion using PIL/Numpy/SciPy today?

Response:
Since 2010 when the linked question was asked the corresponding code moved from scipy to a separate toolkit: http://scikit-image.org/ So here's the code I was actually looking for:
```
from skimage import io, color
rgb = io.imread(filename)
lab = color.rgb2lab(rgb)
```
It should also be noted that due to Lab nature srgb->lab conversion depends on an additional parameter: whitepoint, eg: • Photoshop uses a white point called D50 (which is a standard for icc) • OpenCV and skimage use D65 (which is a standard for srgb). • default Matlab implementation uses D50 (it is capable of using others), This nice FAQ explains it this way: You should use D65 unless you have a good reason to use something else. The print industry commonly uses D50 and photography commonly uses D55. These represent compromises between the conditions of indoor (tungsten) and daylight viewing. You can tell which whitepoint you're dealing with by converting RGB (0,0,255) to Lab: • D50 would give you (30, 68, -112) • D55 (30, 73, -110) • D65 (32, 79, -108) The numbers after 'D' correspond to (internally) used color temperature of white point: D50 = 5003 K (yellowish), D65 = 6504 K (blueish) I'm grateful to Alex and Roman for their answers because they pointed me into the right direction.


Question:
How to generate equispaced interpolating values I have a list of (x,y) values that are not uniformly spaced. Here is the archive used in this question. I am able to interpolate between the values but what I get are not equispaced interpolating points. Here's what I do:
```
x_data = [0.613,0.615,0.615,...]
y_data = [5.919,5.349,5.413,...]

# Interpolate values for x and y.
t = np.linspace(0, 1, len(x_data))
t2 = np.linspace(0, 1, 100)
# One-dimensional linear interpolation.
x2 = np.interp(t2, t, x_data)
y2 = np.interp(t2, t, y_data)

# Plot x,y data.
plt.scatter(x_data, y_data, marker='o', color='k', s=40, lw=0.)

# Plot interpolated points.
plt.scatter(x2, y2, marker='o', color='r', s=10, lw=0.5)
```
Which results in: As can be seen, the red dots are closer together in sections of the graph where the original points distribution is denser. I need a way to generate the interpolated points equispaced in x, y according to a given step value (say 0.1) As askewchan correctly points out, when I mean "equispaced in x, y" I mean that two consecutive interpolated points in the curve should be distanced from each other (euclidean straight line distance) by the same value. I tried unubtu's answer and it works well for smooth curves but seems to break for not so smooth ones: This happens because the code calculates the point distance in an euclidean way instead of directly over the curve and I need the distance over the curve to be the same between points. Can this issue be worked around somehow?

Response:
Convert your xy-data to a parametrized curve, i.e. calculate all all distances between the points and generate the coordinates on the curve by cumulative summing. Then interpolate the x- and y-coordinates independently with respect to the new coordinates.
```
import numpy as np
from matplotlib import pyplot as plt

data = '''0.615   5.349
    0.615   5.413
    0.617   6.674
    0.617   6.616
    0.63    7.418
    0.642   7.809
    0.648   8.04
    0.673   8.789
    0.695   9.45
    0.712   9.825
    0.734   10.265
    0.748   10.516
    0.764   10.782
    0.775   10.979
    0.783   11.1
    0.808   11.479
    0.849   11.951
    0.899   12.295
    0.951   12.537
    0.972   12.675
    1.038   12.937
    1.098   13.173
    1.162   13.464
    1.228   13.789
    1.294   14.126
    1.363   14.518
    1.441   14.969
    1.545   15.538
    1.64    16.071
    1.765   16.7
    1.904   17.484
    2.027   18.36
    2.123   19.235
    2.149   19.655
    2.172   20.096
    2.198   20.528
    2.221   20.945
    2.265   21.352
    2.312   21.76
    2.365   22.228
    2.401   22.836
    2.477   23.804'''

data = np.array([line.split() for line in data.split('\n')],dtype=float)

x,y = data.T
xd = np.diff(x)
yd = np.diff(y)
dist = np.sqrt(xd**2+yd**2)
u = np.cumsum(dist)
u = np.hstack([[0],u])

t = np.linspace(0,u.max(),10)
xn = np.interp(t, u, x)
yn = np.interp(t, u, y)

f = plt.figure()
ax = f.add_subplot(111)
ax.set_aspect('equal')
ax.plot(x,y,'o', alpha=0.3)
ax.plot(xn,yn,'ro', markersize=8)
ax.set_xlim(0,5)
```


Question:
Stop Scipy minimize after set time I use minimize from the Scipy module on Python 3.4, specifically:
```
resultats=minimize(margin_rate, iniprices, method='SLSQP',
jac=margin_rate_deriv, bounds=pricebounds, options={'disp': True,
'maxiter':2000}, callback=iter_report_margin_rate)
```
The maximum number of iterations can be set (as above), but is there a way to tell minimize to stop searching for a solution after a given set time? I looked at the general options of minimize as well as the specific options of the SLSQP solver, but could not work it out. Thanks

Response:
You can use the callback argument to raise a warning or exception if the execution time exceeds some threshold:
```
import numpy as np
from scipy.optimize import minimize, rosen
import time
import warnings

class TookTooLong(Warning):
    pass

class MinimizeStopper(object):
    def __init__(self, max_sec=60):
        self.max_sec = max_sec
        self.start = time.time()
    def __call__(self, xk=None):
        elapsed = time.time() - self.start
        if elapsed > self.max_sec:
            warnings.warn("Terminating optimization: time limit reached",
                          TookTooLong)
        else:
            # you might want to report other stuff here
            print("Elapsed: %.3f sec" % elapsed)

# example usage
x0 = [1.3, 0.7, 0.8, 1.9, 1.2]
res = minimize(rosen, x0, method='Nelder-Mead', callback=MinimizeStopper(1E-3))
```


Question: retrieve misclassified documents using scikitlearn I'm interested to know if there's built in functions in scikitlearn python module, that can retrieve misclassified documents. it's simple i usually write it myself by comparing both predicted and Test vectors and retrieve the documents from the test document array. but i'm asking if there's a built in functionality for it instead of copying the functionality in each python code i write.

Response:
If you have a list of true labels y_test for a set of documents, e.g. ["ham", "spam", "spam", "ham"] and you convert that to a NumPy array, then you can compare it with the predictions in a one-liner:
```
import numpy as np

y_test = np.asarray(y_test)
misclassified = np.where(y_test != clf.predict(X_test))
```
Now misclassified is an array of indices into X_test. @eickenberg is right, this kind of stuff is not implemented in scikit-learn because users are expected to be familiar enough with NumPy to do it themselves in a few lines of code.


Question: Python baseline correction library I am currently working with some Raman Spectra data, and I am trying to correct my data caused by florescence skewing. Take a look at the graph below: I am pretty close to achieving what I want. As you can see, I am trying to fit a polynomial in all my data whereas I should really just be fitting a polynomial at the local minimas. Ideally I would want to have a polynomial fitting which when subtracted from my original data would result in something like this: Are there any built in libs that does this already? If not, any simple algorithm one can recommend for me?

Response:
I found an answer to my question, just sharing for everyone who stumbles upon this. There is an algorithm called "Asymmetric Least Squares Smoothing" by P. Eilers and H. Boelens in 2005. The paper is free and you can find it on google.
```
def baseline_als(y, lam, p, niter=10):
  L = len(y)
  D = sparse.csc_matrix(np.diff(np.eye(L), 2))
  w = np.ones(L)
  for i in xrange(niter):
    W = sparse.spdiags(w, 0, L, L)
    Z = W + lam * D.dot(D.transpose())
    z = spsolve(Z, w*y)
    w = p * (y > z) + (1-p) * (y < z)
  return z
```


Question: What is the difference between scipy.integrate.odeint and scipy.integrate.ode? I have read the docs for these here, but it just states that odeint is "an integrator with a simpler interface based on lsoda from ODEPACK" What is the actual difference? Under what circumstances is using one more appropriate than the other?

Response: If you can solve your problem with odeint, I would recommend that. It is simply a function that integrates an ode using lsoda. If you want a lot more options, ode is a class with many methods and solvers: scipy.integrate.ode scipy.integrate.ode.integrate scipy.integrate.ode.set_f_params scipy.integrate.ode.set_initial_value scipy.integrate.ode.set_integrator scipy.integrate.ode.set_jac_params scipy.integrate.ode.set_solout scipy.integrate.ode.successful


Question: Truncated multivariate normal in SciPy? I'm trying to automate a process that at some point needs to draw samples from a truncated multivariate normal. That is, it's a normal multivariate normal distribution (i.e. Gaussian) but the variables are constrained to a cuboid. My given inputs are the mean and covariance of the full multivariate normal but I need samples in my box. Up to now, I'd just been rejecting samples outside the box and resampling as necessary, but I'm starting to find that my process sometimes gives me (a) large covariances and (b) means that are close to the edges. These two events conspire against the speed of my system. So what I'd like to do is sample the distribution correctly in the first place. Googling led only to this discussion or the truncnorm distribution in scipy.stats. The former is inconclusive and the latter seems to be for one variable. Is there any native multivariate truncated normal? And is it going to be any better than rejecting samples, or should I do something smarter? I'm going to start working on my own solution, which would be to rotate the untruncated Gaussian to it's principal axes (with an SVD decomposition or something), use a product of truncated Gaussians to sample the distribution, then rotate that sample back, and reject/resample as necessary. If the truncated sampling is more efficient, I think this should sample the desired distribution faster.

Response:
I have reimplemented an algorithm which does not depend on MCMC but creates independent and identically distributed (iid) samples from the truncated multivariate normal distribution. Having iid samples can be very useful! I used to also use emcee as described in the answer by Warrick, but for convergence the number of samples needed exploded in higher dimensions, making it impractical for my use case. The algorithm was introduced by Botev (2016) and uses an accept-reject algorithm based on minimax exponential tilting. It was originally implemented in MATLAB but reimplementing it for Python increased the performance significantly compared to running it using the MATLAB engine in Python. It also works well and is fast at higher dimensions. The code is available at: https://github.com/brunzema/truncated-mvn-sampler. An Example:
```
d = 10  # dimensions

# random mu and cov
mu = np.random.rand(d)
cov = 0.5 - np.random.rand(d ** 2).reshape((d, d))
cov = np.triu(cov)
cov += cov.T - np.diag(cov.diagonal())
cov = np.dot(cov, cov)

# constraints
lb = np.zeros_like(mu) - 1
ub = np.ones_like(mu) * np.inf

# create truncated normal and sample from it
n_samples = 100000
tmvn = TruncatedMVN(mu, cov, lb, ub)
samples = tmvn.sample(n_samples)
```
Plotting the first dimension results in: Reference: Botev, Z. I., (2016), The normal law under linear restrictions: simulation and estimation via minimax tilting, Journal of the Royal Statistical Society Series B, 79, issue 1, p. 125-148


Question: Scipy.optimize: how to restrict argument values I'm trying to use scipy.optimize functions to find a global minimum of a complicated function with several arguments. scipy.optimize.minimize seems to do the job best of all, namely, the 'Nelder-Mead' method. However, it tends to go to the areas out of arguments' domain (to assign negative values to arguments that can only be positive) and thus returns an error in such cases. Is there a way to restrict the arguments' bounds within the scipy.optimize.minimize function itself? Or maybe within other scipy.optimize functions? I've found the following advice: When the parameters fall out of the admissible range, return a wildly huge number (far from the data to be fitted). This will (hopefully) penalize this choice of parameters so much that curve_fit will settle on some other admissible set of parameters as optimal. given in this previous answer, but the procedure will take a lot of computational time in my case.

Response:
The minimize function has a bounds parameter which can be used to restrict the bounds for each variable when using the L-BFGS-B, TNC, COBYLA or SLSQP methods. For example,
```
import scipy.optimize as optimize

fun = lambda x: (x[0] - 1)**2 + (x[1] - 2.5)**2
res = optimize.minimize(fun, (2, 0), method='TNC', tol=1e-10)
print(res.x)
# [ 1.          2.49999999]

bnds = ((0.25, 0.75), (0, 2.0))
res = optimize.minimize(fun, (2, 0), method='TNC', bounds=bnds, tol=1e-10)
print(res.x)
# [ 0.75  2.  ]
```


Question:
How to specify upper and lower limits when using numpy.random.normal I want to be able to pick values from a normal distribution that only ever fall between 0 and 1. In some cases I want to be able to basically just return a completely random distribution, and in other cases I want to return values that fall in the shape of a gaussian. At the moment I am using the following function:
```
def blockedgauss(mu,sigma):
    while True:
        numb = random.gauss(mu,sigma)
        if (numb > 0 and numb < 1):
            break
    return numb
```
It picks a value from a normal distribution, then discards it if it falls outside of the range 0 to 1, but I feel like there must be a better way of doing this.

Response:
It sounds like you want a truncated normal distribution. Using scipy, you could use scipy.stats.truncnorm to generate random variates from such a distribution:
```
import matplotlib.pyplot as plt
import scipy.stats as stats

lower, upper = 3.5, 6
mu, sigma = 5, 0.7
X = stats.truncnorm(
    (lower - mu) / sigma, (upper - mu) / sigma, loc=mu, scale=sigma)
N = stats.norm(loc=mu, scale=sigma)

fig, ax = plt.subplots(2, sharex=True)
ax[0].hist(X.rvs(10000), normed=True)
ax[1].hist(N.rvs(10000), normed=True)
plt.show()
```
The top figure shows the truncated normal distribution, the lower figure shows the normal distribution with the same mean mu and standard deviation sigma.


Question: Python: Calculate Voronoi Tesselation from Scipy's Delaunay Triangulation in 3D I have about 50,000 data points in 3D on which I have run scipy.spatial.Delaunay from the new scipy (I'm using 0.10) which gives me a very useful triangulation. Based on: http://en.wikipedia.org/wiki/Delaunay_triangulation (section "Relationship with the Voronoi diagram") ...I was wondering if there is an easy way to get to the "dual graph" of this triangulation, which is the Voronoi Tesselation. Any clues? My searching around on this seems to show no pre-built in scipy functions, which I find almost strange! Thanks, Edward

Response:
The adjacency information can be found in the neighbors attribute of the Delaunay object. Unfortunately, the code does not expose the circumcenters to the user at the moment, so you'll have to recompute those yourself. Also, the Voronoi edges that extend to infinity are not directly obtained in this way. It's still probably possible, but needs some more thinking.
```
import numpy as np
from scipy.spatial import Delaunay

points = np.random.rand(30, 2)
tri = Delaunay(points)

p = tri.points[tri.vertices]

# Triangle vertices
A = p[:,0,:].T
B = p[:,1,:].T
C = p[:,2,:].T

# See http://en.wikipedia.org/wiki/Circumscribed_circle#Circumscribed_circles_of_triangles
# The following is just a direct transcription of the formula there
a = A - C
b = B - C

def dot2(u, v):
    return u[0]*v[0] + u[1]*v[1]

def cross2(u, v, w):
    """u x (v x w)"""
    return dot2(u, w)*v - dot2(u, v)*w

def ncross2(u, v):
    """|| u x v ||^2"""
    return sq2(u)*sq2(v) - dot2(u, v)**2

def sq2(u):
    return dot2(u, u)

cc = cross2(sq2(a) * b - sq2(b) * a, a, b) / (2*ncross2(a, b)) + C

# Grab the Voronoi edges
vc = cc[:,tri.neighbors]
vc[:,tri.neighbors == -1] = np.nan # edges at infinity, plotting those would need more work...

lines = []
lines.extend(zip(cc.T, vc[:,:,0].T))
lines.extend(zip(cc.T, vc[:,:,1].T))
lines.extend(zip(cc.T, vc[:,:,2].T))

# Plot it
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection

lines = LineCollection(lines, edgecolor='k')

plt.hold(1)
plt.plot(points[:,0], points[:,1], '.')
plt.plot(cc[0], cc[1], '*')
plt.gca().add_collection(lines)
plt.axis('equal')
plt.xlim(-0.1, 1.1)
plt.ylim(-0.1, 1.1)
plt.show()
```