Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 17 16:34
    basnijholt commented #351
  • Jun 17 16:34

    basnijholt on master

    Fix class name issue with moder… (compare)

  • Jun 17 16:34
    basnijholt closed #351
  • Jun 17 14:58
    maiani opened #351
  • Jun 01 18:06
    basnijholt commented #344
  • May 31 22:20
    basnijholt synchronize #244
  • May 31 22:20

    basnijholt on mypy

    rename job Merge remote-tracking branch 'o… (compare)

  • May 31 22:20
    basnijholt synchronize #244
  • May 31 22:20

    basnijholt on mypy

    use typing-extensions (compare)

  • May 31 22:14

    basnijholt on master

    rename job (compare)

  • May 31 20:27

    basnijholt on v0.13.2

    (compare)

  • May 31 18:20
    basnijholt synchronize #244
  • May 31 18:20

    basnijholt on mypy

    use mamba specify file in conda tag add cloudpickle and loky and 4 more (compare)

  • May 31 17:56

    basnijholt on typeguard-job

    (compare)

  • May 31 17:56

    basnijholt on master

    Run seperate typeguard job (bec… (compare)

  • May 31 17:56
    basnijholt closed #350
  • May 31 17:36
    codecov-commenter commented #350
  • May 31 17:32
    basnijholt closed #344
  • May 31 17:32

    basnijholt on master

    set loop to None in Python 3.10… (compare)

  • May 31 17:32

    basnijholt on py310-loop

    (compare)

Adel Kara Slimane
@AdelKS
For example, calculate an additional 10 000 points
or follow a new goal function, while keeping the existing data
kwant-bot
@kwant-bot
anton:
Do you mean the learner?
anton:
The runner is only responsible for communicating with the computation.
anton:
In the learner, you can set a new goal.
Adel Kara Slimane
@AdelKS
Oh, I think I understand better, the data is stored in the learner and I can just make a new runner out of the existing learnertill it finishes again ?
(Sorry for the newbie questions... )
kwant-bot
@kwant-bot
anton:
Yep, exactly. No problem at all.
Adel Kara Slimane
@AdelKS
Thanks!
Adel Kara Slimane
@AdelKS
Hello again, can you please point me to code examples without Jupyter if possible ? I am doing something wrong where the runner has the status running but apparently not making any calculation:
class AdaptiveCalculation:
    def __init__(self, tqdot) -> None:
        self.tqdot = tqdot
        self.useful_data_target_size = 0
        self.useful_data = set()
        self.data_to_process_for_st1 = set()
        self.intermediary_subdiv = 20

        T_amp = 150
        ΔT_amp = 150
        Δμ_amp = 150
        V0_amp = 150

        self.learner_st0 = adaptive.LearnerND(self.tqdot.efficiency_adaptive_st0, bounds=[(0, T_amp), (0, ΔT_amp), (0, Δμ_amp), (-V0_amp/2, V0_amp/2)])

    def useful_points_goal():
        print("Useful points goal called")
        for sub_sim_settings, eff0 in self.learner_st0.data.items():   
            if eff0 > 0 and sub_sim_settings not in self.useful_data:
                self.useful_data.append(sub_sim_settings)
                self.data_to_process_for_st1.append(sub_sim_settings)
        return len(self.useful_data) > self.useful_data_target_size

    def start_adaptive_coverage(self, useful_data_target_size = -1):

        executor = ProcessPoolExecutor(max_workers=4)

        self.useful_data_target_size = useful_data_target_size
        runner = adaptive.Runner(self.learner_st0, goal=self.useful_points_goal, executor=executor)

        while len(self.useful_data) > self.useful_data_target_size:
            print("Runner status: " + runner.status())
            print("Elapsed time: {}".format(runner.elapsed_time()))
            print("Learner points number: {}".format(len(self.learner_st0.data)))
            print("Number of useful points: {}".format(len(self.useful_data)))

            if len(self.data_to_process_for_st1) > self.intermediary_subdiv:
                print("intermediary processing data")
                runner.cancel()
                # Do something here
                print("Picking back up adaptive coverage")
                runner = adaptive.Runner(self.learner_st0, goal=self.useful_points_goal, executor=executor)

            sleep(2)


if __name__ == '__main__':
    tqdot = tQdot_adaptive()
    coverer = AdaptiveCalculation(tqdot)
    coverer.start_adaptive_coverage()
Bas Nijholt
@basnijholt
@AdelKS, there are no examples to run Adaptive in scripts. However, it shouldn't be any different from using it in a notebook, except for the fact that you cannot use the notebook widgets (obviously).
In a script you can either use the async/await formulism to await runner.task or just use the adaptive.BlockingRunner.
The code you wrote wouldn't work because useful_points_goal doesn't take the learner.
The signature needs to be goal(learner: adaptive.BaseLearner) -> float: pass.
Bas Nijholt
@basnijholt
You could probably use a closure to create your goal. Something like
def get_useful_points_goal(n_useful_points):
    useful_data = []
    def useful_points_goal(learner):
        for sub_sim_settings, eff0 in learner.data.items():   
            if eff0 > 0 and sub_sim_settings not in useful_data:
                useful_data.append(sub_sim_settings)
        return len(useful_data) > n_useful_points
    return useful_points_goal
kwant-bot
@kwant-bot
martenarthers:
I use adaptive without notebook in my code, you could check out run_adap() function as an example. Unfortunatly, it does not open a separate window to show live results. It just evaluates your function until your set goal is reached, and then continues you script afterwards. The learner.data attribute is returned.
martenarthers:
in the same repo, you check out the runner.py to see how I use this function, and how the data is plotted eventually
Adel Kara Slimane
@AdelKS
@basnijholt thank you for pointing out the mistake with the goal function signature, and @martenarthers thank you for your code example. That should be enough ressources for me to do what I want, thanks!
Adel Kara Slimane
@AdelKS
@martenarthers unfortunately the link you provided doesn't work for me :thought_balloon:
kwant-bot
@kwant-bot

martenarthers:
Ah right. Well here is the function that I use. (not sure if it will help you, but you can always have a look):

def run_adap(function, bounds: tuple, client, goal, loss_func=None, nr_tries=15):
    learner = adaptive.Learner2D(function, bounds=bounds, loss_per_triangle=loss_func)

    tries = 0
    while True:
        tries += 1
        try:
            runner = adaptive.Runner(learner, executor=client, goal=goal)
            runner.ioloop.run_until_complete(runner.task)
            print(f'try {tries} succeeded')
            break
        except Exception as e:
            if runner.status() == 'failed':
                print(f'try {tries} failed')
                print(str(e))
                if tries >= nr_tries:
                    raise
            else:
                raise
        time.sleep(5)
    return learner.data

The loop I use is just because the cluster client I use starts up slowly and so it only works after a few tries. The lines that are in principle needed to make it work are:

learner = adaptive.Learner2D(function, bounds=bounds)
runner = adaptive.Runner(learner, goal=goal)
runner.ioloop.run_until_complete(runner.task)
data = learner.data
martenarthers:
I then plot this data with matplotlib. It is also possible to still use the adaptive plotters after runner.ioloop.run_until_complete(runner.task), I have no experience doing this but I think it is similar to how it is done in the notebook examples
Adel Kara Slimane
@AdelKS
noted ! thanks!
quaeritis
@quaeritis
Hello is it possible to use adaptive.Runner with ipyparallel.Client and the cloudpickle support of ipyparallel (dview.use_cloudpickle())?
kwant-bot
@kwant-bot
anton:
Hmm, I suspect it should work directly if you specify use_cloudpickle before starting the simulation
quaeritis
@quaeritis

o.k that was too easy, embarrassing.

dview = client[:]
dview.use_cloudpickle()

I didn't think that dview.use_cloudpickle() would affect client as well, thanks.

kwant-bot
@kwant-bot
anton:
👍
antoaert
@antoaert
Hello!
First, thank you so much for this tool! I plan to use it to construct a potential energy surface to perform quantum dynamics computations :)
Not really an issues, but something unfortunate happens when I use adaptive on multiple cores combined with a goal based on a number of points.
In this case, let's say I run 32 calls to the function I want to sample simultaneously (use 32 cores) but my current goal is for instance ">=2", then the "learner.data.values()" contains only 2 values.
It is easy to work around it but one could easily end up wasting ressources if he/she doesn't realize.
kwant-bot
@kwant-bot
basnijholt:
You are either wasting resources or time. It might be that one of those 32 points is returned faster than others.
kwant-bot
@kwant-bot
basnijholt:
Hi there, that looks cool but it might not be super intuitive to use. From the code I understand that you add a function bounds which can be called with the combo, right?
Joschka Winkel
@MeisterWinkel_gitlab
This questing has probably already been asked (if yes, could you link it?): How do I get the data from the learner such that I can plot it with matplotlib? Thanks, j.winkel
Anton Akhmerov
@akhmerov
learner.data
Anton Akhmerov
@akhmerov
That will contain learner-dependent data, typically the coorinates of the points requested and the returned values
Joschka Winkel
@MeisterWinkel_gitlab
Thanks, that was fast. One question that remains for me is how to run the runner in python3. runner.live_info() and runner.live_plot(update_interval=0.1) seem to require jupyter
I dont need to see the live plot, just the results
Anton Akhmerov
@akhmerov
An alternative that uses an AsyncRunner can use the example here, so that you run runner.ioloop.run_until_complete(runner.task).
Joschka Winkel
@MeisterWinkel_gitlab
Now it works! Thanks a lot
Joschka Winkel
@MeisterWinkel_gitlab
Is it possible to ignore grid-points which evaluated to NaN's such that they get removed and those points not tried again?
Anton Akhmerov
@akhmerov
Not straightforwardly, or at least I'm not sure how to do so.
What's your use case?
Anton Akhmerov
@akhmerov
If it's OK with you, please open a github issue so that we can track this feature. Please also provide some motivation for your use case—that'd be useful as well.
Joschka Winkel
@MeisterWinkel_gitlab
My problem was that the 2D function i wanted to calculate one a 2D sheet sometimes evaluated to NaN near interesting places since I had a zero division inside some integral (not problematic, the integral was still finite) and I wanted these NaNs to be ignored to make an interpolation easier. I solved the problem now by slightly changing the integrand such that the NaNs disappear (not the most elegant solution, but convenient), but thank you for your readyness to help.
emascot
@emascot
Hello, I'm currently using Learner1D on a function that uses scipy.optimize.root. Do you have any suggestions for handling when the algorithm doesn't converge? Is there a way to filter out unsuccessful points from the loss function?
Anton Akhmerov
@akhmerov
Hah, that's an interesting and useful question!
OK, looking at the previous messages, it's exactly the same question :P
So the answer is that I don't think it easily works out of the box, but it's definitely something that we'd want to support.
emascot
@emascot
Ok, thanks. resolution_loss_function works for me for now.
gctomaz
@gctomaz
Hi, I am using LearnerND with a 3D function. I would like to use only integers for one parameter, while still using real numbers for the others. Is there a way I can define that? Thank you!
Anton Akhmerov
@akhmerov
Uh, tough call. I don't think it works out of the box