Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 11:46
    ManfreddEE opened #768
  • 09:40
    nesaboz edited #767
  • 09:39
    nesaboz opened #767
  • Jan 19 07:19
    zspitz commented #246
  • Jan 18 20:10
    slozier commented #763
  • Jan 18 20:10
    slozier labeled #763
  • Jan 18 19:49
    slozier commented #246
  • Jan 18 19:44
    slozier commented #765
  • Jan 18 19:23
    zspitz commented #246
  • Jan 18 18:39
    slozier labeled #765
  • Jan 18 18:11
    slozier labeled #766
  • Jan 18 14:45
    slozier commented #246
  • Jan 18 12:23
    danabr opened #766
  • Jan 14 23:48
    zspitz commented #246
  • Jan 14 23:46
    zspitz commented #246
  • Jan 13 00:48
    ridgew commented #701
  • Jan 12 03:25
    BCSharp commented #1085
  • Jan 12 03:24
    BCSharp edited #1086
  • Jan 12 03:06
    BCSharp opened #1086
  • Jan 09 08:39
    jol5 commented #763
Alex Earl
@slide
??
slozier
@slozier
If they (Microsoft) have any plans for IronPython then they haven't reached out
Alex Earl
@slide
@slozier I ran this test https://github.com/pythonnet/pythonnet/blob/e0f47ba41a4a11e88efc6f17aa1d0647943c8108/src/embed_tests/TestNamedArguments.cs after modifying the Test3 method in the python code to use the * keyword only syntax and it worked
doh
well, it won't let me edit that
class cmTest3:
    def Test3(self, a1 = 1, *, a2 = 1, a3 = 1, a4 = 1):
        return a1 + a2 + a3 + a4

a = cmTest3()
there
slozier
@slozier
Cool, so looks like you just have to explicitly specify the name in your dynamic call or use Py.kw
Alex Earl
@slide
right
Alex Earl
@slide
@slozier how are things going for you these days
?
slozier
@slozier
Not bad, kids are getting older and not waking up as often so I get more sleep. :)
ipy3 is also coming along well, it's got better test coverage that ipy2 so once I find time to figure out that nonlocal business I'll be able to release something.
Pavel Koneski
@BCSharp
@slozier is there anything specific I can look into, nonlocal or otherwise, to help with the release? So far I have been primarily looking into stuff from the "Text vs. data" section in WhatsNewInPython3.0 (and various fun spin-offs), but having a release would be a nice thing
slozier
@slozier
@BCSharp If you'd like to take a stab at nonlocal that would be a big help, it's basically the major blocker right now. I did mark up a few things which would be nice for a release (https://github.com/IronLanguages/ironpython3/milestone/1).
Alex Earl
@slide
nonlocal was a pain!
Pavel Koneski
@BCSharp
OK, will give nonlocal a shot after I wrap up the DLR excursion
darpant-289
@darpant-289
Exception thrown: 'IronPython.Runtime.Exceptions.ImportException' in Microsoft.Dynamic.dll
ddelapasse2
@ddelapasse2
Hi Everyone! We've been using IronPython with good results, but today have hit a use case where we need numpy and pandas. Is that possible?
of course I have googled, but all the responses I find are 7-8 years old
Alex Earl
@slide
numpy and pandas use native libraries, which IronPython doesn't support.
ddelapasse2
@ddelapasse2
@slide OK thanks...it's some pretty simple 2d matrix stuff so hopefully we can find another way.
Alex Earl
@slide
There is a library that implements some of the numpy operations in .net that could be wrapped with a similar api
slozier
@slozier
ddelapasse2
@ddelapasse2
thanks guys! I will check out both
Dunge
@Dunge

Hello there, just chiming in in hope someone might have a magical solution to my situation.

I have a C# application (running as a Windows service on a server) whose sole role is to execute python scripts. At startup it loads a bunch of scripts from the database, create a python engine, compile them, and then on a timer loop (every minute) run them all, each on a separate thread using Task.Run(). Each of these threads create a Scope, inject a C# class using Scope.SetVariable() then call compiledCode.Execute(scope). The python scripts themselves all do some very minor and very quick CPU logic, and then call a method on the "variable" class which is back in C#. The thing is, this method is doing network-bound operation, and currently is calling an asynchronous method synchronously (using .wait()).

Now my issue is that while the C# network operation is run, the python code is waiting and blocking a CPU thread preventing the other scripts to run. At low load it doesn't show any symptoms, but when running over 800 scripts per minute, I get the feeling the thread pool is getting starved and the network-bound operations are preventing all my scripts to execute fast enough during that minute loop, which cause a queue to overflow somewhere. In pure C# you would just await the network operation and other threads could be handled in the meantime, but using IronPyton there's no "ExecuteAsync" method, and of course the await keyword doesn't exists Python, so I'm left to having one executing python blocking the thread waiting for the other network operation thread to complete which seems like a complete waste of processor resource.

Is there any method I could optimize it to scale my processing power in a more intelligent way and not reach a bottleneck of having too many scripts preventing it to run smoothly?

slozier
@slozier
Maybe I'm missing something, but instead of having the python call the method on C# class why not do that step on the C# side of things and use the async/await goodness?
Dunge
@Dunge
well in reality we offers a few methods to the users of the python script, some yes are just post-actions (I could run them async after the script), but some are used to real data (ie. from redis, backing to database if not cached) and the parameters depends on what the script compute, and needs to be feeded back to the script for further processing
slide-o-mix
@slide-o-mix:matrix.org
[m]
An ExecuteAsync might be interesting
Dunge
@Dunge
real = read (sorry typo)
I know all the cpu processing done in python is extra light, I could process 1000x more in the allowed minute. I also know all network operations would have ample time to complete if done in parallel. But now what I observe is that only a few (4/8?) scripts get executed in parallel, and the others are waiting for one to complete before starting, which make them slowly process one by one, and all that probably due to the network wait
slozier
@slozier
Not sure how an ExecuteAsync would work beyond doing Task.Run. As for the number of scripts that are executing in parallel, that's probably limited by Task.Run which uses the default thread pool and is initially limited by the number of cores on the machine (which is why you might see 4/8). You could try messing with SetMinThreads (https://docs.microsoft.com/en-us/dotnet/api/system.threading.threadpool.setminthreads) to let more go through...
Dunge
@Dunge
Yeah I've read that you better not to mess with the min thread count, it will just destabilize the OS who will spend its time switching between them because physically the number is limited anyway. My question was more to see if there was somehow a way to gain the advantage that async/await gives in C# of the thread pool context switching when they are waiting for IO bound operations when run from a Python thread execution context.. but since the python code is run synchronously I already knew that was a no.
No matter what the thread will be used from the start to the end of the python execution
elTRexx
@elTRexx

Hi, happy christmas everyone. :)
So I came back coding again, and now that I have a "dynamic" variable from my .py script wich I know it is a function (thx to @slozier answer back in a days : if (o is PythonFunction)), can I get more intel from this "dynamic" variable ? I'm trying to figure out for example how to get the MethodInfo of this PythonFunction "dynamic" variable via Reflection ??
I was looking over some ironpython source and googled. Came across this for exemple : https://social.msdn.microsoft.com/Forums/vstudio/en-US/59f88a7d-fa56-4e9f-95ac-d9e7e3bb8aed/how-to-get-ironpython-method-parameters-using-net?forum=csharpgeneral
or that : https://stackoverflow.com/questions/31658951/how-do-i-extract-the-names-of-ironpython-script-arguments-in-c-sharp
But this is kinda a workaround using reflection to get the builtin ironpython ArgNames private proterty, ich only give the ... arguments Names (could get the extra return variable name via varNames too i Guess). Am I stick only with strings intel ? Couldn't I figure out somahow the given C# type of return values / parameters of a given python function in my .py script ?

Thanks for any help again. ;)

slozier
@slozier
Hmm, the problem is that none of the arguments or the return value of a Python function are strongly typed, they're basically all of type object. I'm not sure how you could do anything more than figuring out the number of arguments and their names (which you can probably do without reflection via the FunctionCode object).
elTRexx
@elTRexx
Just trying to think out loud (well, in a chat) :
  • let's say I have this simple python script :
    #test.py
    def f(a,b):
      return a+b
  • then in C# side
    int a = 2;
    int b = 5;
    ScriptEngine engine = Python.CreateEngine();
    ScriptScope scope = engine.CreateScope();
    engine.ExecuteFile(@"C:\test.py", scope);
    dynamic cs_F = scope.GetVariable("f");
    dynamic result = cs_F(a,b);
  • then could I do this ?
    Console.WriteLine("the actual return type of my python function in C# is :" + result.GetType());
slozier
@slozier
Sure, assuming your functions are always simple that might work. But that assumes you know your inputs are integers? If you give it floats (or strings, lists, etc.) instead then your return type changes.
slozier
@slozier
Only thing I can think of is to annotate your function with the expected types. Maybe with a function decorator?
ddelapasse2
@ddelapasse2
We do json.dumps(returnVar) in python - then in C# when we deserialize it we can pretty much get the return type (at least string vs numeric)
developer-ken
@developer-ken
Nice to meet you guys
I wonder how to set a timeout for IronPython

So something like

while True:
    print("a")

won't block everything

This is my code at present:

py = Python.CreateEngine();
scope = py.CreateScope();
py.CreateScriptSourceFromString(code).Execute<object>(scope);

Or how do I kill a script when it is running? Maybe I could use a timer and kill the script when it runs for too long.

ddelapasse2
@ddelapasse2
Happy New Year All! I was wondering if anyone here creates a python editing screen inside an Angular project? We have been trying Monaco/Monarch but having a really hard time so wondering if there are any other openSource editors we should be trying instead that more readily can handle python2.7 so we can use with IronPython?
slozier
@slozier
@developer-ken Don't know of a good way to do that. With .NET Framework you could run in a thread and then call Thread.Abort, but unfortunately that does not work with .NET Core...
elTRexx
@elTRexx

Hi again. Happy new Year. Sorry I wasn't active last days.
What I'm up for my way to get the return type of any given python function within C# world is something like that

// something like the above except that any function I'm looking for are all parameter-less
dynamic cs_func = scope.GetVariable("func");
MethodInfo methodInfo = new MethodUtils<Action>(funcVar);
return methodInfo.ReturnType;

Where MethodUtils is the methodOf taken from this : https://stackoverflow.com/questions/14329433/get-methodinfo-for-any-method-with-any-signature-delegate-for-any-signature

using System;
using System.Reflection;

/// <summary>
/// Utility class to get a correponding MethodInfo (System.Reflection) of a given function,
/// using T as a delegate (Action<> e.g.) representation/"encapsulation" of the C# function
/// ! care, Action<???> as T have to be strongly typed. !
/// </summary>
/// <typeparam name="T">T is the function encapsulated within a Delegate (Action<>)</typeparam>
public class MethodUtils<T>
{
    public MethodUtils(T func)
    {
        var del = func as Delegate;
        if (del == null) throw new ArgumentException("Cannot convert func to Delegate.", "func");

        Method = del.Method;
    }

    private MethodInfo Method { get; set; }

    public static implicit operator MethodUtils<T>(T func)
    {
        return new MethodUtils<T>(func);
    }

    public static implicit operator MethodInfo(MethodUtils<T> methodOf)
    {
        return methodOf.Method;
    }
}

So using some kind on encapsulation of the ironpython C# func of the python function within a Delegate to get the MethodInfo of the Delegate and ultimately get the ReturnType property of the MethodInfo class

slozier
@slozier
Cool, only problem with that is if your method is defined in Python it'll always have an return type of System.Object I think
Although if it's not the case I'd love to know what the type is :smile: