Napi api: Batch API

  • GeSHi library error: sites/all/modules/geshifilter/geshi is not a directory.
  • The spam filter installed on this site is currently unavailable. Per site policy, we are unable to accept new submissions until that problem is resolved. Please try resubmitting the form in a couple of minutes.
Category: 
Leírás
Napi api: Batch API

Well, talán harmadjára futok össze a batch apival, és még mindig nem volt zökkenőmentes a barátság, szóval amíg az a betyár sok elem leszalad, amin dolgoztam, levésem gyorsan a tudnivalókat, hogy legközelebb már ne kelljen megint mindenre rájönni.

No akkor batch api:

A batch kötegelt feldolgozást jelent. Drupalon belül úgy is mondhatjuk, hogyha egyszerre annyi adattal kell dolgozni, ami túlnő egy oldalletöltés korlátjain, akkor kell a batch apihoz nyúlni. A képet megnézve valószínű egyébként, hogy egyből leesik, a Drupal melyik részéről is van szó.

Nézzük a feladatot: Adott a favorite_nodes modul, D7 migráció során flaget kell belőle faragni több 10k nagyságrendű adatból.

A batch api maga két fő részből áll. Van egyszer egy batch definíció, és minimum egy operation callback, az a funkció, amely a valódi műveletet fogja elvégezni.

Így néz ki nagyjából egy batch definíció:

[geshifilter-drupal6] $batch = array( 'operations' => array( array('operation_callback', array()) ), 'finished' => 'favorite_nodes_to_flag_batch_finished', 'title' => t('Convert all favorite nodes to flag'), 'init_message' => t('Convert starting..'), 'progress_message' => t('Processed @current out of @total.'), 'error_message' => t('Hmm.. Something could be wrong, sorry, post an issue to module..'), );[/geshifilter-drupal6]

Az operations elem alatt lehet felsorolni azokat a függvényeket, amelyeket sorban hívni fog a drupal egymás után. Az 'operation_callback' utáni array() most üres, de segítségével paramétereket adhatunk át vele a függvényünknek. Mivel általában batch processt form_submitkor indítunk, átadhatunk például form értékeket.

A 'finished' elemben egy olyan függvényt írhatunk, amely a batch művelet legvégén fut le. Például üzeneteket (drupal_set_message()) írhatunk ki, hogy mi történt a folyamat során. Figyi, ez nem array, ebből csak egy van!

Az 'init_message' általában nem látszik sokáig, nagyjából amíg összerakja drupal a fejében, hogy mit és hányszor fog csinálni - ezzel indul az egész folyamat.

Az 'error_message', amit senki nem akar látni, többnyire több egyéb nyalánkság, főleg pdo exception társaságában fogjuk viszontlátni.

El is érkeztünk, hogy hogy is indul el egy batch process: Tipikusan form_submit részben összerakjuk a batch arrayt, majd egy batch_set függvénnyel kirúgjuk a batchába, a többit intézi a drupal magának:

[geshifilter-drupal6]function my_module_form_submit($form, &$form_state) { $batch = array(lásd felül); batch_set($batch); }[/geshifilter-drupal6]

Ezzel elkezdődik az operation része a batchnek. Ez a függvény annyiszor fog lefutni, ahányszor kell (már ha jól van megírva..)
Az operation callbackünk legfontosabb része egy $context nevű paraméter, ez az, amelyben a komplett művelet során tudunk változókat átadni, vezetni, módosítani az egész folyamat során, vagyis ha $context['sandbox']['counter']++ -t teszünk a függvénybe, az minden egyes alkalommal növekedni fog, ahányszor a függvény meghívódik a batch process során.

Ennek a $context paraméternek a következő részeit állítgathatjuk:

$context['results'] - egy tömbbe gyűjthetünk adatokat, amelyekkel közvetlen azután dolgozhatunk, hogy lefutott a batch. Ezt elérhetjük abban a függvényben, amit a $batch['finished']-nél adtunk meg.

$context['sandbox'] - tükörfordításban ugye homokozó, én jobban szeretem a szemétdombot. Itt összegyűjthetünk, módosíthatunk minden olyan változót, amivel függvényünk dolgozik. Például ahogy fentebb is volt már, hogy hányszor futott a process.

$context['message'] - lásd kép, ez az a szöveg, ami folyamatosan frissül a böngészőben, ahányszor az operation elindul.

$context['finished'] - Hogy hány %-nál jár a művelet. Arra még nem sikerült rájönnöm, hogy vajon ez vezérli-e az egész batch process futását, de sejtésem szerint igen. Szóval egészen addig fog újra és újra futni az operation, amíg itt el nem éri a 100%-ot - a tudományom mai állása szerint.

Nézzünk egy ilyen függvényt, és magyarázzunk:

[geshifilter-drupal6]function MODULOM_batch_operation(&$context) { // Az első futásnál ez a $context['sandbox'] üres // Ilyenkor kell feltölteni inicializálni pár dolgot if (empty($context['sandbox'])) { // Ez lesz most a számlálónk, vagyis hogy hány elemet dolgoztunk fel. $context['sandbox']['progress'] = 0; // Ez az ütköző: Maximum ennyi elem feldolgozását végezzük el a teljes batch process során $context['sandbox']['max'] = db_query('SELECT COUNT(nid) FROM {favorite_nodes}')->fetchField(); // Ezt majd üzenetgyártásra fogom használni, számláló ahhoz, hogy flaget sikerült legyártani. $context['success'] = 0; } // Ennyi elemen végzünk művelete egyszerre, egy futás alatt. $limit = 70; // Egy mezei query, összegyűjtjük, hogy ezen futás során épp mely elemekkel dolgozunk. // Figyi erre: ->range($context['sandbox']['progress'], $limit), látszik, hogy progress értéket használjuk offsetként a queryben, tehát épp onnantól kezdődik az új eredményhalmazunk, ahol az előző futáskor abbahagytuk. $result = db_select('favorite_nodes', 'f') ->fields('f', array('nid', 'uid', 'last')) ->range($context['sandbox']['progress'], $limit) ->execute(); // Most dolgozunk az eredményhalmazon. foreach ($result as $row) { // Először növeljük a progresst. $context['sandbox']['progress']++; // Beállítjuk, az üzenetjelzőben, hogy épp hányadik elemet bántjuk $context['message'] = $context['sandbox']['progress'] . ' item processed.'; // Ezek saját vuduk, létrehozzuk a flaget, és beállítjuk a results-ban, hogy sikeresült-e létrehozni a flaget, vagy nem if (_favorite_nodes_to_flag_flag($row)) { $context['success']++; $context['results']['success'][] = t('Success: User id: @uid, Node id: @nid', array( '@uid' => $row->uid, '@nid' => $row->nid )); } else { $context['results']['failed'][] = t('Flagging failed: User id: @uid, Node id: @nid', array( '@uid' => $row->uid, '@nid' => $row->nid )); } } // Na ez tűnik a critical partnak, ahogy emlegettem. Ha itt benézünk valamit, akkor lesz egy foreverliving batch processünk. Szóval figyeljünk, hogy hogyan állítjuk be az ütközőt. if ($context['sandbox']['progress'] != $context['sandbox']['max']) { $context['finished'] = $context['sandbox']['progress'] / $context['sandbox']['max']; } } [/geshifilter-drupal6]
Érdemes batch gyártásnál először a keretet elkészíteni, esetleg egy hülyebiztos lekérdezéssel húzni az időd műveletenként, hogy pontosan úgy fut-e, azon az elemeken, amikkel nekünk kell. Az írási-módosítási részeket ráérünk akkor, amikor már úgy fut a batch, ahogy kell.

Ritka szemét dolog, amikor 30 perc futás után 90%+ processnél elhasal valami hülyeségen a mutatvány..

Comments

Excellent blog you have here

Excellent blog you have here but I was curious about if you knew
of any community forums that cover the same topics discussed
in this article? I'd really love to be a part of community where
I can get suggestions from other experienced individuals that share the same interest.
If you have any suggestions, please let me know. Thanks!

I am in fact glad to read

I am in fact glad to read this blog posts which consists of plenty of valuable information, thanks for
providing these kinds of data.

I enjoy reading through an

I enjoy reading through an article that can make people think.
Also, thanks for allowing for me to comment!

Article writing is also a

Article writing is also a fun, if you be acquainted with
after that you can write otherwise it is complex to write.

Self-actuating

Self-actuating wmt.zeqs.szantogabor.com.wyu.bb skill, [URL=http://fountainheadapartmentsma.com/cialis-super-active/ - lowest price generic cialis super active canada[/URL - [URL=http://center4family.com/pharmacy/ - pharmacy[/URL - [URL=http://kitsunesbooks.com/nitrofurantoin/ - nitrofurantoin uk price comparison[/URL - [URL=http://davincipictures.com/levitra-generic-overnight-shipping/ - levitra[/URL - levitra for sale online cpx24 [URL=http://mrcpromotions.com/cialis-5-mg/ - q online cialis[/URL - mirror listing valproate, lowest price generic cialis super active canada canadian pharmacy price nitrofurantoin nitrofurantoin en genericos levitra discount levitra 10 pack generic cialis no percription burnishing nadir principles http://fountainheadapartmentsma.com/cialis-super-active/ comprare cialis super active farmacia http://center4family.com/pharmacy/ pharmacy http://kitsunesbooks.com/nitrofurantoin/ buy nitrofurantoin low prise http://davincipictures.com/levitra-generic-overnight-shipping/ achats de pilules levitra http://mrcpromotions.com/cialis-5-mg/ cialisonline tissue strengths, abort 1-4%.

Heⅼpo thеre, Yooս have done

Heⅼpo thеre, Yooս have done a great job. I'll definitely digg it and personally recommend to my friends.
I am sure they will be benefited from this wweb site. https://adsfree.mastermindswadd.site/index.php?page=user&action=pub_prof...

We stumbled over here by a

We stumbled over here by a different page and thought I may
as well check things out. I like what I see so now i'm
following you. Look forward to looking into your web page repeatedly.

find out

Wonderful beat ! I would like

Wonderful beat ! I would like to apprentice while you amend your website,
how could i subscribe for a blog site? The account helped me a
acceptable deal. I had been tiny bit acquainted of this your broadcast offered bright clear concept

I’m not that much of a online

I’m not that much of a online reaⅾer to be honest but your sites reɑlly nice, қeep it up!
I'll go ahead and booкmark your site tto сome bawck later.
Cheers http://bpbdkaltim.com/

I'm pretty pleased to uncover

I'm pretty pleased to uncover this page.
I need to to thank you for ones time just for this wonderful read!!
I definitely loved every little bit of it and i also have you saved as
a favorite to check out new information on your web site.

I hɑve гeadd so many ϲontent

I hɑve гeadd so many ϲontent regardіng tthe blogger lovers
but this post iis truly ɑ fаstidious piece of wгiting, keep it up. http://zxgmb.com/comment/html/?389945.html

I think the admin of this web

I think the admin of this web page is in fact working hard in favor
of his web page, for the reason that here every material is quality based stuff.

I savor, lead to I found just

I savor, lead to I found just what I was taking a look for.
You have ended my four day long hunt! God Bless you man. Have a great day.
Bye

Hi Dear, are you actually

Hi Dear, are you actually visiting this web page regularly, if so then you will absolutely obtain fastidious experience.

A zománc fürdők

If you are going for best

If you are going for best contents like I do, only pay a visit this web page all the time since it offers feature contents,
thanks

As I website possessor I

As I website possessor I believe the content material here is
rattling excellent , appreciate it for your hard work.
You should keep it up forever! Best of luck.

I constantly spent my half an

I constantly spent my half an hour to read this weblog's content everyday along with
a cup of coffee.

Me recomendaram este web

Me recomendaram este web site e me disseram que o assunto era sempre conversado
em alto nível. Acessei e pude comprovar exatamente isto.
Muito bom pelo artigo! https://tinyurl.com/yyc3katd

먹튀사이트 검증사이트 먹튀폴리스입니다. 먹튀폴리스는

먹튀사이트 검증사이트 먹튀폴리스입니다. 먹튀폴리스는 신규는 물론 오픈된 먹튀사이트를 철저히 먹튀검증하여 먹튀없는 시대를 만들어가고 있습니다.
먹튀폴리스를 사랑하는 여러분의 많은 호응바랍니다.
여러분의 충실한 먹튀검증 사이트가 되도록 노력하겠습니다.

Remаrkable issues here. I'm

Remаrkable issues here. I'm very glad to peer your article.
Thanks ѕo much and I'm looking ahead too touch you.
Will you kindly drop me a mail? https://narsingdihaat.com/index.php?page=user&action=pub_profile&id=352665

Hi it's me, I am also

Hi it's me, I am also visiting this site daily, this website is in fact pleasant and the users are really sharing fastidious thoughts.

Actually when someone doesn't

Actually when someone doesn't understand afterward its
up to other visitors that they will assist, so here it takes place.

At this time I am going to do

At this time I am going to do my breakfast, afterward having
my breakfast coming over again to read additional news.

Woah! I'm really enjoying the

Woah! I'm really enjoying the template/theme of this site.
It's simple, yet effective. A lot of times it's tough to get
that "perfect balance" between usability and appearance. I must say
you have done a excellent job with this. Also,
the blog loads extremely quick for me on Opera. Exceptional Blog!

Marvelous, what a blog it is!

Marvelous, what a blog it is! This website presents useful
data to us, keep it up.

Medicines information sheet.

Medicines information sheet. Drug Class. https://lisinopril2020.top Everything about drugs. Get information here.

Wow, superb blog format! How

Wow, superb blog format! How lengthy have you been blogging for?

you made blogging glance easy. The entire glance of your site is wonderful,
as neatly as the content! https://www.flagsonastick.com/4th-of-july-garden-flags/

яндекс сео продвижение

яндекс сео продвижение

Hello, I check your blog on a

Hello, I check your blog on a regular basis. Your
story-telling style is awesome, keep it up!

консультація юриста

консультація юриста

стоимостьадвоката запорожье

I bⅼoց quite often and I

I bⅼoց quite often and I truly thank you for your information. This article has
realⅼy peake myу interest. І will book mark your weƅsite and
keep checking for neew details about oncе a week. I subscribsd to your Feed as
well. https://nigmart.com/user/profile/397141

Гидра сайт

I love it whenever people get

I love it whenever people get together and share ideas.
Great site, stick with it!

Hi, I ⅾo believe thiѕ iis аn

Hi, I ⅾo believe thiѕ iis аn excellent wеbsite. Ι stumbledupon іt ;) I
may return ʏet again since і have bookmarked it. Μoney and freedom is the greatest way to change, may you be rich and continue to help others. http://vipwww.vip/comment/html/?87872.html

It's an remarkable post in

It's an remarkable post in favor of all the web users; they will obtain benefit from
it I am sure.

It's аwеsome to visit this

It's аwеsome to visit this website and reading the
views of alll mates concerning thiѕ post, while I am also keen of
getting knowledge. http://vipwww.vip/comment/html/?86935.html

excellent submit, very

excellent submit, very informative. I'm wondering why the
opposite experts of this sector do not notice this.
You must continue your writing. I'm sure,
you have a great readers' base already!

Hey! Do you know if they make

Hey! Do you know if they make any plugins to help with Search Engine Optimization?
I'm trying to get my blog to rank for some targeted keywords but I'm not seeing very good
success. If you know of any please share. Thank you!

My spouse and I stumbled

My spouse and I stumbled over here different page and thought I
might as well check things out. I like what I see so now i'm following you.
Look forward to finding out about your web page again.

It's the best time to make a

It's the best time to make a few plans for the longer term and it's time to be happy.

I've learn this publish and if I may I wish to suggest you some attention-grabbing issues or tips.

Maybe you could write subsequent articles relating to this article.
I want to read even more things approximately it!

fiberglass insect screening

Hello there! I know this is

Hello there! I know this is kinda off topic but I was
wondering which blog platform are you using for this site? I'm getting sick and tired of Wordpress because I've had issues with hackers and I'm looking at
options for another platform. I would be awesome if you could point
me in the direction of a good platform.

Ahaa, its fastidious dialogue

Ahaa, its fastidious dialogue about this piece of writing here at this webpage, I have read all that,
so at this time me also commenting here.

Whаt'ѕ up, just wanted to

Whаt'ѕ up, just wanted to mention, I loved this article.
It was inspiring. Kеep on posting! http://ejournal.umm.ac.id/index.php/legality/comment/view/19/0/101893

Hi, every time i used to

Hi, every time i used to check web site posts here early in the dawn, as i enjoy to find out more and
more.

It's goіng to be end of mine

It's goіng to be end of mine day, but beforе end I am
reading thiis gгeat рiece of writing to improve my experience. http://hitommy.net/xe/SoccerWorld/4113777

Hey There. I found your

Hey There. I found your weblog the usage of msn. That is an extremely smartly written article.

I will make sure to bookmark it and return to learn extra of your useful information. Thanks for the post.
I will definitely return.

Cotton Canvas Pencil Pouch

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Internal paths in double quotes, written as "internal:node/99", for example, are replaced with the appropriate absolute URL or relative path.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <del> <img>
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <blockquote> <br> <cite> <code> <dd> <del> <div> <dl> <dt> <em> <li> <ol> <p> <span> <strong> <ul>
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <mysql>, <php>, <python>, <ruby>, <sql>. The supported tag styles are: <foo>, [foo].
  • Each email address will be obfuscated in a human readable fashion or (if JavaScript is enabled) replaced with a spamproof clickable link.